Sequence Recognition with Recurrent Neural Networks
Arun Maskara, Andrew S. Noetzel · Connection Science · 1993
The simple recurrent network (SRN) introduced by Elman (1990) can be trained to predict each successive symbol of any sequence in a particular language, and thus act as a recognizer of the language. Here, we show several conditions occurring within the class of regular languages that result in recognition failure by any SRN with a limited number of nodes in the hidden layer. Simulation experiments show how modified versions of the SRN can overcome these failure conditions. In one case, it is found to be necessary to train the SRN to show at its output units both the current input symbol as well as the predicted symbol. In another case, the SRN must show the current contents of the context units. It is shown that the SRN with both modifications, called the auto-associative recurrent network (AARN), overcomes the identified conditions for SRN failure, even when they occur simultaneously. However, it cannot be trained to recognize all of the regular languages.