ASCOC: a recurrent neural network model for grammatical inference
Dit Yan Yeung, Kei-wai Yeung · 1994
Many real-world problems require the handling of temporally correlated events. Recently, some discrete-time recurrent neural network models have been proposed to solve various temporal sequence processing problems. One of the most popular models is Elman's simple recurrent network (SRN), which has been used for grammatical inference (or formal language learning) problems. In this paper, some major drawbacks of SRN and its variants are identified and investigated in detail. Based on these observations, a new model called ASCOC, which stands for auto-associative SRN with context-to-output connections, is proposed to overcome the problems encountered by SRN. Two experiments in using the ASCOC to learn regular grammars are reported, with detailed analysis of the hidden layer patterns learned by the network. Not only is it easier to train and use ASCOC than SRN, the new model is also more powerful in that it can remember more nonlocal information that is relevant to correct learning of gram...