Memory Convergence Analysis for Different Iterative Procedures in RNN
Subhash C. Pandey · 2010
Recurrent neural networks (RNN) are a class of densely connected single layer nonlinear networks of perceptrons. The network’s energy function is defined through a learning procedure so that its minima coincide with states from a predefined set.. However, because of the network’s nonlinearity a number of undesirable local energy minima emerge from the learning procedure. This has shown to significantly effect the network’s performance. In this work we analyze the rate of convergence for three iterative procedures namely- Mann, Ishikawa and J-iterations in recurrent network and many important results have been worked out for decreasing as well as increasing functions. The results obtained are very useful for designing of inner product kernel of support vector machine with faster convergence rate.