Autonomous learning algorithm for fully connected recurrent networks.
Edouard Leclercq, Fabrice Druaux, Dimitri Lefebvre · 2003
Abstract:. In this paper a fully connected RTRL neural network is studied. In order to learn dynamical behaviours of linear-processes or to predict time series, an autonomous learning algorithm has been developed. The originality of this method consists of the gradient based adaptation of the learning rate and time parameter of the neurons using a small perturbations method. Starting from zero initial conditions (neural states, rate of learning, time parameter and matrix of weights) the evolution is completely driven by the dynamic of the learning data. Two examples are proposed, the first one deals with the learning of second order linear process and the second one with the prediction of the chaotic intensity of NH3 laser. This last example illustrates how our network is able to follow high frequencies. Recurrent neural networks are very helpful to solve dynamical problems. Properties of dynamical recurrent networks such as oscillatory or chaotic behaviours were studied between 1989 and 1995 [1, 2]. Recently, various applications have been