Arbitrary sensitive transitions in recurrent neural networks
Muhammed Fadera, Peter Ashwin · Physica D Nonlinear Phenomena · 2024
An Excitable Network Attractor (ENA) is a forward-invariant set in phase space that can be used to explain input-driven behaviour of Recurrent Neural Networks (RNNs) trained on tasks involving switching between a discrete set of states. An ENA is composed of two or more attractors and excitable connections that allow transitions from one attractor to another under some input perturbation. The smallest such perturbation that makes a connection between two attractors is called the excitability threshold associated with that connection. The excitability threshold provides a measure of sensitivity of the connection to input perturbations. Errors in performance of such trained RNNs can be related to errors in transitions around the associated ENA. Previous work has demonstrated that ENAs of arbitrary sensitivity and structure can be realised in a RNN by suitable choice of connection weights and nonlinear activation function. In this paper we show that ENAs of arbitrary sensitivity and structure can be realised even using a suitable fixed nonlinear activation function, i.e. by suitable choice of weights only. We show that there is a choice of weights such that the probability of erroneous transitions is very small. • In this paper we show that Excitable Network Attractors can be realised using a suitable fixed nonlinear activation function and choice of weights only. • The realisation allows arbitrary sensitivity and graph structure. • We show that for some choices of the parameters, the set of initial conditions that leaves the network has zero measure. • Using bifurcation analysis and stochastic simulations, we show that this realisation can be extended to Excitable Network Attractors of periodic orbits.