The second derivative of a recurrent network
S. Piche · 1994
The equations for the exact calculation of the second derivative of an error function with respect to the weights (Hessian matrix) of a recurrent network are presented in this paper. The second derivative of feedforward networks has proven useful for fast retraining, weight pruning, and output error estimation. However, until now, techniques based upon the Hessian could not be used for recurrent networks because no exact equations for the second derivative existed. It is the author's hope that the equations presented which allow for the exact calculation of the second derivative will prove useful in the development of new methods for designing recurrent networks.>