Recurrent Neural Networks Can Learn to Implement Symbol-Sensitive Counting
Paul Rodriguez, Janet Wiles · 1997
Recently researchers have derived formal complexity analysis of analog computation in the setting of discrete-time dynamical systems. As an empirical constrast, training recurrent neural networks (RNNs) produces self-organized systems that are realizations of analog mechanisms. Pre-vious work showed that a RNN can learn to process a simple context-free language (CFL) by counting. Herein, we extend that work to show that a RNN can learn a harder CFL, a simple palindrome, by organizing its re-sources into a symbol-sensitive counting solution, and we provide a dy-namical systems analysis which demonstrates how the network: can not only count, but also copy and store counting infonnation.