Robust algorithm for neural network training
Milos Manic, Bogdan M. Wilamowski · 2003
Neural networks have been proven to be very successful in many cases where other traditional techniques failed to give satisfactory results. Despite their popularity, several problems exist. Even with the adequate network architecture, frustrating problems of correct choice of initial weights for given architecture remain. The proposed method uses combination of approaches used in genetic algorithms and gradient methods. Genetic algorithm is used in search for an adequate weight set for a complex error surface. Once it is done, the algorithm automatically shifts to gradient type of method. The proposed algorithm does not explicitly calculate gradients like in error back propagation. It rather estimates the gradient from the set of random feed forward calculations. The proposed approach automatically searches for the adequate initial weight set. This robustness with respect to initial weight set is achieved through introduction of randomness in neuron weight space. Results are confirmed through experimental data and given in the form of tables and graphs.