ASCOC : A Locally Recurrent Neural Network Model for Grammatical Inference
Di Yan Yeung, Kai Wai Yeung · 1994
Many real-world problems require the handling of temporally related events. Recently, some recurrent neural network models were proposed to solve various temporal sequence processing problems. One of the most popular models is Elman's simple recurrent network (SRN), which has been used for formal language learning problems. In this paper, some major drawbacks of SRN are identified and investigated in detail. Based on these observations, a new model called ASCOC that overcomes the problems encountered by SRN is proposed. Two experiments in using the ASCOC to learn regular grammars are reported, with detailed analysis of the hidden layer patterns learned by the network. Not only is it easier to train and use ASCOC than SRN, the new model is also more powerful in the sense that it can remember more nonlocal information that is relevant to correct learning of grammars with embedded structures. Possible extension to domains beyond formal language learning is also discussed. 1 Introduction ...