An accelerated recurrent network training algorithm
Amir F. Atiya, A.G. Parlos · 2002
There have been extensive efforts to develop training algorithms for recurrent neural networks. A variety of algorithms have been developed, but still recurrent network training is plagued by slow convergence. The goal of this paper is to develop a new algorithm that is based on approximating the direction of the error gradient. The new algorithm has lower computational complexity in computing the weight update than the competing techniques for most typical problems. In addition, it reaches the error minimum in a much smaller number of iterations. Typically, it reaches the minimum within only about 5 or 10 iterations, compared to around a 1000 iterations or so for the competing techniques.