Dynamics of structural learning with an adaptive forgetting rate
DAMON A. MILLER, Jacek M. Żurada · Proceedings of International Conference on Neural Networks (ICNN'97) · 2002
Structural learning with forgetting is a prominent method of multilayer feedforward neural network complexity regularization. The level of regularization is controlled by a parameter known as the forgetting rate. The goal of this paper is to establish a dynamical system framework for the study of structural learning both to offer new insights into this methodology and to potentially provide a means of either developing new or analytically justifying existing forgetting rate adaptation strategies. The resulting nonlinear model of structural learning is analyzed by developing a general linearized equation for the case of a quadratic error function. This analysis demonstrates the effectiveness of an adaptive forgetting rate. A simple example is provided to illustrate our approach.