Comparison of Neural and Statistical Classifiers - Theory and Practice
Lasse Holmström, Petri Koistinen, Jorma T. Laaksonen, Erkki Oja · 1996
Pattern classification using neural networks and statistical methods is discussed. We first give a tutorial overview that groups popular classifiers according to their underlying mathematical principles into several distinct categories. Starting from the Bayes classifier, one division is whether the classifier is explicitly estimating class conditional densities, or directly estimating the posterior probabilities by regression. Another criterion is the flexibility of the architecture in the sense of how rich the discriminant function family is. Still one dimension is neural vs. nonneural learning: neural learning is characterized by simple local computations in a number of real or virtual processing elements. Based on these comparisons, a number of classification methods were selected for a case study that uses handwritten digit data. An effort was made to get fair estimates of their true classification performance, thus training set cross-validation was extensively used to design the various classifiers. The classification errors were estimated with an independent testing set. The performance of a number of most typical neural and statistical classifiers was compared. Also, four methods of our own were used in the comparisons: the Reduced Kernel Discriminant Analysis (RKDA), the Learning k-