Neural network model of temporal pattern memory
Kenji Doya, Shuji Yoshizawa · Systems and Computers in Japan · 1991
Abstract This paper proposes a neural network model which memorizes temporal patterns such as movement patterns of animals. A transient or periodic signal waveform is memorized and regenerated as a transient response waveform or a waveform of an autonomous oscillation in the continuous‐time neural network with recurrent connections. The back‐propagation (BP) learning rule, which has been used primarily in the discrete‐time neuron model with feed‐forward connections, is extended in two ways so that it is applicable to the continuous‐time neural model with recurrent connections. As a result of computer simulation, it is shown that “direct BP learning,” which considers only the direct effect of the output of the hidden unit on the output unit, has almost the same learning performance with less computation time than “simultaneous BP learning,” which is based more strictly on the learning by the steepest descent.