Improving neural network performance for character and fingerprint classification by altering network dynamics
Charles L. Wilson, James L. Blue, Omid M. Omidvar · 1995
In previous work [1], the Probabilistic Neural Network, PNN,[2] was shown to provide better zero-reject error performance on character and fingerprint classification problems than Radial Basis Function, RBF, and Multilayer Perceptron, MLP, based neural network methods.Later work [3] demonstrated that various combined neural networks could provide performance equal to PNN and substantially better error-reject performance but these systems were very expensive to train and were much slower and less memory efficient than MLP based systems.In this paper, we will show that performance equal to or better than PNN can be achieved with a single three-layer MLP by making fundamental changes in the network optimization strategy.These changes are: 1) Neuron activation functions are used which reduce the probability of singular Jacobians; 2) Successive regularization is used to constrain volume of the weight space being minimized; 3) Boltzmann pruning is used [4] to constrain the dimension of the weight space; and 4) Prior class probabilities are used to normalize all error calculations so that statistically significant samples of rare but important classes can be included without distortion of the error surface.All four of these changes are made in the inner loop of a conjugate gradient optimization iteration [5] and are intended to simplify the training dynamics of the optimization.On handprinted digits and fingerprint classification problems these modifications improve error-reject performance by factors between 2 and 4 and reduce network size by 40% to 60%.