Generating exponentially many periodic attractors with linearly growing Echo State Networks
Herbert Jaeger · 2006
When a human listens to a novel piece of music, wherein a reasonably short motif (likewise novel to the listener) is repeated twice, the human is able to (1) detect this fact, (2) continue to reproduce the motif periodically. A similar phenomenon is iterated phone number rehearsal to keep a phone number in short-term memory. Mathematically, these are cases where a dynamical system (the brain) hosts a huge number of periodic attractors, into which it can be driven by feeding the corresponding periodic time series as a cue input. Here we demonstrate how this phenomenon can be modelled using Echo State Networks featuring a spatial encoding of musical pitch. The network is trained as a pure delay-line memory. The number of periodic attractors that a given, trained ESN can produce without further parameter adjustment scales exponentially with the network size. A stability analysis of the attractors is provided.