Creating Attractors Using BPTT Algorithm with Recurrent Neural Networks
Haruhiko Takase, Kazutoshi Gohara, Y. Uchikawa · Transactions of the Society of Instrument and Control Engineers · 1996
We have already demonstrated that, by regarding the time variable external input of finite length as one cycle of the wave form of a periodic function, RNN's can be described as forced vibration systems. It was also pointed out that to explain behaviors of RNNs we must consider attractors created in the state space of RNNs corresponding to training patterns. This paper describes our recent systematic simulations to investigate into behaviors of additive nets, typical models of RNNs, In the case where multiple combinations of simple wave forms of limited length are employed as time sequential training patterns. The results showed that (1) a cyclic trajectory is created in the state space of the RNN corresponding to each of training patterns, and (2) transition time from one cyclic trajectory to another induced by switching from one input pattern to another can be shortened remarkably by selecting appropriate combinations of training patterns.