Estimation of artificial neural network parameters for nonlinear system identification
Timothy L. Ruchti, Ronald H. Brown, J.J. Garside · 2005
A unified framework for representing ANN (artificial neural network) training algorithms is developed by considering weight selection as a parameter estimation problem. Three existing ANN training strategies are reviewed within this framework, i.e., gradient-descent backpropagation, the extended Kalman algorithm, and the recursive least squares method. A strikingly different approach to error backpropagation is presented, resulting in the development of a novel method of backward signal propagation and target state generation for embedded layers. The proposed technique is suitable for implementation with a linear-Kalman based update algorithm and is applied with a time-varying method of covariance modification for the elimination of transients associated with initial conditions. Results from a nonlinear identification experiment demonstrate an increased rate of convergence in comparison with backpropagation. The new algorithm displayed similar rates of parameter convergence and a decreased computational overhead compared to the extended Kalman algorithm.>