Karhunen Loève feature extraction for neural handwritten character recognition
Patrick J. Grother · Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE · 1992
The optimality of the Karhunen Loeve (KL) transform is well known. Since its basis is the eigenvector set of the covariance matrix, a statistical, not functional, representation of the variance in pattern ensembles is generated. By using the KL transform coefficients as a natural feature representation of a character image, the eigenvector set can be regarded as an unsupervised biological feature extractor for a (neural) classifier. The covariance matrix and its eigenvectors are obtained from 76,753 handwritten digits. This operation is a unique expense; once the basis set is calculated it forms a linear first layer of a three weight layer feed forward network. The subsequent nonlinear perceptron layers are trained using a scaled conjugate gradient algorithm that typically affords an order of magnitude reduction in computation over the ubiquitous back-propagation algorithm. In conjunction with a massively parallel computer, training is expedited such that tens of initially different random weight sets are trained and evaluated. Increase in training set size (up to 76,755 patterns) gives less accurate learning but improved generalization on the fixed disjoint test set. A neural classifier is realized that recognizes 96.1% of 15,000 handwritten digits from 944 different writers. This recognition is attributed to the energy compaction optimality of the KL transform.