Convergence of Online Gradient Method for Recurrent Neural Networks
Xiaoshuai Ding, Ruiting Zhang · Journal of Interdisciplinary Mathematics · 2015
Recurrent neural networks are widely used for analysis and prediction of temporal sequences. An online gradient learning method for recurrent neural networks is considered and its convergence is proved in this paper. Rather than most of the convergence results are of probabilistic nature under the assumption that a great number of training samples of temporal series are available, the convergence theorem we built here is deterministic nature and is founded on the premise that merely a limited number of training samples are furnished. The monotonicity of the error function during the training iteration is guaranteed as well.