BIDIRECTIONAL HIERARCHICAL NEURAL NETWORKS - Hebbian Learning Improves Generalization
Mohammad Saifullah, Rita Kovordányi, Chandan Roy · 2010
Visual pattern recognition is a complex problem, and it has proven difficult to achieve satisfactorily instandard three-layer feed-forward artificial neural networks. For this reason, an increasing number ofresearchers are using networks whose architecture resembles the human visual system. These biologicallybasednetworks are bidirectionally connected, use receptive fields, and have a hierarchical structure, withthe input layer being the largest layer, and consecutive layers getting increasingly smaller. These networksare large and complex, and therefore run a risk of getting overfitted during learning, especially if smalltraining sets are used, and if the input patterns are noisy. Many data sets, such as, for example, handwrittencharacters, are intrinsically noisy. The problem of overfitting is aggravated by the tendency of error-drivenlearning in large networks to treat all variations in the noisy input as significant. However, there is one wayto balance off this tendency to overfit, and that is to use a mixture of learning algorithms. In this study, weran systematic tests on handwritten character recognition, where we compared generalization performanceusing a mixture of Hebbian learning and error-driven learning with generalization performance using pureerror-driven learning. Our results indicate that injecting even a small amount of Hebbian learning, 0.01 %,significantly improves the generalization performance of the network.