On the nonlinearity of pattern classifiers
Aarnoud Hoekstra, Robert P. W. Duin · 1996
This paper presents a novel approach to the analysis of the overtraining phenomenon in pattern classifiers. A nonlinearity measure /spl Nscr/ is introduced which relates the shape of the classification function to the generalization capability of a classifier. Experiments using the k-nearest neighbour rule, a neural classifier and the quadratic classifier show that the introduced measure /spl Nscr/ can be used to study the overtraining behaviour of a classifier. Moreover /spl Nscr/ shows to be a predictor for the local sensitivity of a classifier. Classifiers that have a small local sensitivity are shown to have a low nonlinearity whereas an increased nonlinearity indicates an increase in local sensitivity.