Knowledge extraction and recurrent neural networks
Ingo Schellhammer, Joachim Diederich, Michael Towsey, Claudia Brugman · 1998
We present results of experiments with Elman recurrent neural networks (Elman, 1990) trained on a natural language processing task. The task was to learn sequences of word categories in a text derived from a primary school reader. The grammar induced by the network was made explicit by cluster analysis which revealed both the representations formed during learning and enabled the construction of state-transition diagrams representing the grammar. A network initialised with weights based on a prior knowledge of the text's statistics, learned slightly faster than the original network.