Accuracy Barrier (ACCBAR): A novel performance indicator for binary classification
Gürol Canbek, Tuğba Taşkaya Temizel, Şeref Sağıroğlu · 2022
Although several binary classification performance metrics have been defined, a few of them are used for performance evaluation of classifiers and performance comparison/reporting in the literature. Specifically,$\boldsymbol{F}1$and Accuracy$\boldsymbol{ACC}$are the most known and conventionally used metrics. Despite their popularity and easy-to-understand characteristics, those metrics exhibit critical robustness issues. This paper suggests a new instrument category named 'performance indicators' and proposes a novel indicator named accuracy barrier (ACCBAR for short) that works to uncover confounding problems in performance reporting of$\boldsymbol{ACC}$metric. The given case study in mobile malware classification, which is a domain of cyber security, has shown that the indicator gives an accurate interpretation of the results presented in terms of$\boldsymbol{ACC}$, This study also recommends that researchers should use ACCBAR to eliminate potential publication or confirmation bias in classification performance evaluation.