Globally Optimal Parameters for On-Line Learning in Multilayer Neural Networks
David Saad, Magnus Rattray · Physical Review Letters · 1997
We present a framework for calculating globally optimal parameters, within a given time frame, for on-line learning in multilayer neural networks. We demonstrate the capability of this method by computing optimal learning rates in typical learning scenarios. A similar treatment allows one to determine the relevance of related training algorithms based on modifications to the basic gradient descent rule as well as to compare different training methods.