On Obtaining Classification Confidence, Ranked Predictions and AUC with Tsetlin Machines
Kuruge Darshana Abeyrathna, Ole‐Christoffer Granmo, Morten Goodwin · 2020
Tsetlin machines (TMs) are a promising approach to machine learning that uses Tsetlin Automata to produce patterns in propositional logic, leading to binary (hard) classifications. In many applications, however, one needs to know the confidence of classifications, e.g. to facilitate risk management. In this paper, we propose a novel scheme for measuring TM confidence based on the logistic function, calculated from the propositional logic patterns that match the input. We then use this scheme to trade off precision against recall, producing area under receiver operating characteristic curves (AUC) for TMs. Empirically, using four real-world datasets, we show that AUC is a more sensitive measure of TM performance compared to Accuracy. Further, the AUC-based evaluations show that the TM performs on par or better than widely used machine learning algorithms. We thus believe our scheme will make the TM more suitable for use in decision support, where the user needs to inspect and validate predictions, in particular, those being uncertain.