A CLASSICAL ALGORITHM FOR AVOIDING LOCAL MINIMA
Denise Gorse, Adrian J. Shepherd, JG Taylor · 1994
This paper has presented results which violate the widespread belief that the only way to avoid local minima in supervised learning problems with complex error-weight surfaces is to use computationally expensive stochastic procedures like simulated annealing or genetic algorithms. If the results presented here can be shown to be securely founded, and the ERA method shown to have wide applicability, there could be a significant changes in the way that supervised learning tasks are approached. We believe that it is possible to construct a rigorous mathematical proof that the ERA method will work in all but pathological (and rare) cases. The details of this proof are too lengthy to be presented here, but the general principles can be outlined. Initially we look at the first ERA step, for which the homotopy parameter 0 < l << 1. For such small l, the error E(l) of (2) can be expanded as E(l) = E(0) + lE 1 + O(l