Modeling Prediction Uncertainty in Regression using the Regression Tsetlin Machine
Kuruge Darshana Abeyrathna, Andreas Hafver, L. Yi Edward · 2023
Machine learning is being widely used in various industries, and its impact varies based on the application. However, high-risk domains like autonomous vehicles and medical imaging require a higher level of accuracy and uncertainty estimation. Among other approaches, probabilistic deep learning methods have been developed to quantity both aleatoric and epistemic uncertainty by estimating probability distributions for aleatoric variables and fitting distributions to model parameters, respectively. Tsetlin Machines (TMs) are an emerging technology in machine learning known for their fast learning, low energy and memory consumption, high prediction accuracy, and interpretability. While previous research can be modified to identify epistemic and aleatoric uncertainty in classification tasks, it remains unclear how TMs can quantity aleatoric uncertainty in their regression predictions. In this research, we demonstrate how the Regression Tsetlin Machine (RTM) can be adapted to quantity uncertainty in regression tasks by modifying its structure and learning mechanism. Essentially, we divide the clauses into two groups where one group predicts the mean regression output while the other group learns to predict the variance associated with the predicted mean. Learning involves regular RTM learning, however with slight modifications to learn the variance where clauses in this group are trained based on the error of the predicted mean. We evaluate our proposed approach to estimate the uncertainty using artificial data, both when the variance is random and variance is a function of the input. We compare the performance of the proposed method against a deep learning model. The results demonstrate that RTM based model performs on par or better compared to the deep learning model.