Backpropagation using generalized least squares
A.P. Loh, K.F. Fong · 2002
The backpropagation algorithm is essentially a steepest gradient descent type of optimization routine minimizing a quadratic performance index at each step. The backpropagation algorithm is re-cast in the framework of generalized least squares. The main advantage is that it eliminates the need to predict an optimal value for the step size required in the standard backpropagation algorithm. A simulation result on the approximation of a nonlinear dynamical system is presented to show its rapid rate of convergence compared to the backpropagation algorithm.>