How chaos boosts the encoding capacity of small recurrent neural networks : learning consideration
Colin Molter, Utku Salihoglu, Hugues Bersini · 2005
So far, recurrent networks, when adopting fixed point dynamics, show a very poor encoding capacity. However, these same networks, when preferentially maintained in chaotic dynamics, can encode an enormous amount of information in their cyclic attractors and this boosts their encoding capacity. It has been described in a previous paper a simple way to encode such information by robustly associating each vector in a N-dimensional space with one "symbolic" cyclic attractor. The main message was the monotonous increase of chaotic spontaneous regimes as a function of the number of attractors to learn. However, no algorithm was provided to adjust the connection's weight in order to encode a given input set. For this purpose, this paper revisits the classical gradient-based BPTT learning algorithm. It shows that this algorithm gives poor results and furthermore that by using it the "chaoticity" of the network dampens strongly, hence it's encoding capacity.