Karhunen Loeve feature extraction for neural handwritten character recognition
Patrick J. Grother · 1992
The optimality of the Karhunen Loeve (KL) transform is well known.Since its basis is the eigen- vector set of the covariance matrix, a statistical, not functional, representation of the variance in pattern ensembles is generated.By using the KL transform coefficients as a natural feature representation of a character image, the eigenvector set can be regarded as an unsupervised biological feature extractor for a (neural) classifier.The covariance matrix and its eigenvectors are obtained from 76753 handwritten digits.This operation is a unique expense; once the basis set is calculated it forms a linear first layer of a three weight layer feed forward network.The subsequent nonlinear perceptron layers are trained using a scaled conjugate gradient algorithm that typicaUy affords an order of magnitude reduction in computation over the ubiquitous back- propagation algorithm.In conjunction with a massively parallel computer, training is expedited such that tens of initially different random weight sets are trained and evaluated.Increase in training set size (upto 76755 patterns) gives less accurate learning but improved generalization on the fixed disjoint test set.A neural classifier is realized that recognizes 96.1% of 15000 hand- written digits from 944 different writers.This recognition is attributed to the energy compaction optimality of the KL transform.1