An entropy gain measure of numeric prediction performance
Leonard E. Trigg · 1998
Categorical classi er performance is typically evaluated with respect to error rate, expressed as a percentage of test instances that were not correctly classi ed. When a classi er produces multiple classi cations for a test instance, the prediction is counted as incorrect (even if the correct class was one of the predictions). Although commonly used in the literature, error rate is a coarse measure of classi er performance, as it is based only on a single prediction o ered for a test instance. Since many classi ers can produce a class distribution as a prediction, we should use this to provide a better measure of how much information the classi er is extracting from the domain. Numeric classi ers are a relatively new development in machine learning, and as such there is no single performance measure that has become standard. Typically these machine learning schemes predict a single real number for each test instance, and the error between the predicted and actual value is used to calculate a myriad of performance measures such as correlation coe cient, root mean squared error, mean absolute error, relative absolute error, and root relative squared error. With so many performance measures it is di cult to establish an overall performance evaluation. The next section describes a performance measure for machine learning schemes that attempts to overcome the problems with current measures. In addition, the same evaluation measure is used