Memory in Echo State Networks and the Controllability Matrix Rank
Brian Whiteaker, Peter Gerstoft · ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · 2022
Echo State Networks (ESNs) are a variant of recurrent neural networks (RNNs). ESNs perform as nonlinear fading memory filters and excel in prediction of "chaotic" signals. Predictions are made using a forced nonlinear dynamical system called the "reservoir" which incorporates past information into new states. The length of memory is critical to a task effective ESN. We examine the rank behavior of minimal task-effective ESNs predicting the chaotic Lorenz 1963 system for single and multi-variable input/output. Relationships are observed between the rank of the controllability matrix, memory length, and attractor features. We find that reservoir memory varies dependent on input forcing and location in state space. Knowledge of controllability matrix rank indicates a signal specific range for memory length. This variability corresponds to the reservoir varying between stable and unstable. The controllability matrix rank can facilitate efficient use of data and ESN construction.