Classification model evaluation
Paweł Cichosz · 2015
This chapter provides an overview of classifier performance measures and evaluation procedures as well as the general discussion of model evaluation caveats. The purpose of the evaluation of a classification model is to get a reliable assessment of the quality of the target concept's approximation represented by the model, which is called the model's predictive performance. For any performance measure, it is important to distinguish between its value for a particular dataset (dataset performance), especially the training set (training performance), and its expected performance on the whole domain (true performance). One convenient tool that facilitates classifier performance evaluation in multiple operating points, operating point comparison, and operating point selection, is the receiver operating characteristic (ROC) analysis. The importance of model evaluation in the practice of data mining cannot be overestimated, and the classification task is no exception.