Complexity control method for recurrent neural networks
Masao Sakai, Noriyasu Honma, K. Abe · 2003
This paper demonstrates that the Lyapunov exponents of recurrent neural networks can be controlled by our proposed methods. One of the control methods minimizes a squared error e/spl lambda/=(/spl lambda/-/spl lambda//sup obj/)/sup 2//2 by a gradient method, where /spl lambda/ is the largest Lyapunov exponent of the network and /spl lambda//sup obj/ is a desired exponent. /spl lambda/ implying the dynamical complexity is calculated by observing the state transition for a long-term period. This method is, however, computationally expensive for large-scale recurrent networks and the control is unstable for recurrent networks with chaotic dynamics since a gradient correction through time diverges due to the chaotic instability. We also propose an approximation method in order to reduce the computational cost and realize a "stable" control for chaotic networks. The new method is based on a stochastic relation which allows us to calculate the correction through time in a fashion without time evolution. Simulation results show that the approximation method can control the exponent for recurrent networks with chaotic dynamics under a restriction.