Preconditioning method to accelerate neural networks gradient training algorithms
María José Pérez-Ilzarbe · 2003
In this work a simple method for conditioning neural networks gradient training algorithms is presented. It consists of using a different learning rate for the outgoing weights of each one of the neurons or network input nodes. In the case of one layer neural networks the method can also be implemented by normalizing the input training examples in certain way. The performance of the method proposed has been tested in the training of neural networks to solve a problem of image recognition. A considerable acceleration of the training algorithms has been attained in the examples tested.