Two adaptive stepsize rules for gradient descent and their application to the training of feedforward artificial neural networks
Mohamed Ahmed Mohandes, Craig W. Codrington, Saul B. Gelfand · 1994
Gradient descent, in the form of the well-known backpropagation algorithm, is frequently used to train feedforward neural networks, i.e. to find the weights which minimize some error measure /spl epsi/. Generally, the stepsize is fixed, and represents a compromise between stability and speed of convergence. In this paper, we derive two methods for adapting the stepsize and apply them to train neural networks on parity problems of various sizes.>