On the Applicability of Neural Network and Machine Learning Methodologies to Natural Language Processing
Steve Lawrence, Clyde Lee Giles, Sandiway Fong · University Libraries (University of Maryland) · 1998
How can we apply neural network and machine learning methodologies to natural language processing? In this paper we consider the task of training a neural network to classify natural language sentences as grammatical or ungrammatical thereby exhibiting the same kind of discriminatory power provided by the Principles and Parameters linguistic framework, or Government-and-Bindingtheory. We have investigated the following models: feed-forward neural networks, Frasconi-Gori-Soda and Back-Tsoi locally recurrent neural networks, Williams and Zipser and Elman recurrent neural networks, Euclidean and editdistance nearest-neighbors, simulated annealing, and decision trees. Non-neural network machine learning methods are included primarily for comparison. Initial simulations were only partially successful by using a large temporal window as input to the models. Investigation indicated that success obtained this way did not imply that the models had learnt the grammar to a significant degree. Att...