The learning rate in back-propagation systems: an application of Newton's method
R.H. White · 1990
In backpropagation learning, the internode connection strengths, or weights, are adjusted by a method of gradient descent in weight space. The author shows how to apply a multidimensional version of Newton's method for finding the roots of an equation to the question of determining how far to move down the gradient in each learning cycle in backpropagation. The results of a few simulations for a fully recurrent net are presented. The results show an appreciable improvement, by a factor of five to ten, in the convergence rate for these hard-to-learn tests