Accelerated learning in MLP using adaptive learning rate with momentum coefficient
Satya Sheel, Tarun Varshney, R. Varshney · 2007
The ability of a neural network to realize some complex nonlinear function makes them attractive for system identification. In the recent past, neural networks trained with back-propagation (BP) learning algorithm have gained attention for the identification of nonlinear dynamic systems. Slower convergence and longer training times are the disadvantages often mentioned when the standard BP algorithm are compared with other competing techniques. In addition, in the standard BP algorithm, the learning rate is fixed and that it is uniform for all weights in a layer. In this paper, we present an improvement to the standard BP algorithm based on the use of an adaptive learning rate and momentum term, where the learning rate is adjusted at each iteration to reduce the training time. Simulation results indicate a faster convergence speed and better error minimization as compared to other competing methods.