Second differentials in arbitrary feedforward neural networks
Fabrice Rossi · 2002
We extend here a general mathematical model for feedforward neural networks. Such a network is represented as a vectorial function f of two variables, x (the input of the network) and w (the weight vector). We have already shown that the differential of f can be computed with an extended back-propagation algorithm as well as with a direct method. In this paper, we show that the second differentials of f can also be computed with several different algorithms. Evaluating the theoretical complexities of these methods allow one to choose the fastest algorithm for a particular architecture. This will allow us to handle arbitrary feedforward neural network learning with the help of recent training and analysis techniques based on the Hessian matrix of the error.