An Artificial Grammar Investigation into the Mental Encoding of Syntactic Structure
Pyeong Whan Cho, Emily Szkudlarek, Anuenue Kukona, Whitney Tabor · Cognitive Science · 2011
An Artificial Grammar Investigation into the Mental Encoding of Syntactic Structure Pyeong Whan Cho ([email protected]) 1,2 Emily Szkudlarek ([email protected]) 1 Anuenue Kukona ([email protected]) 1,2 Whitney Tabor ([email protected]) 1,2 Department of Psychology and Cognitive Science Program, University of Connecticut, 406 Babbidge Road U-1020, Storrs, CT 06269 USA Haskins Laboratories, 300 George St., Suite 900, New Haven, CT 06511 USA Abstract We explore neural network learning and parallel human learning on an artificial language task. The task generates rich data on human interaction with syntactic systems, including recursive ones. Studying the network’s properties, we argue for a “Structured Manifold” view of syntactic representation. The “Structured Manifold” lies in the parameter space (weight space) of the network. It exhibits (1) loci of high order, corresponding to complex rule systems, (2) continuity, which explains how one rule system can morph into another one, and (3) “recursion approximation”, a concept related to symbolic recursion, which addresses some of the puzzles about embedding patterns in human behavior. Keywords: artificial grammar learning; artificial neural networks; recurrent networks; simple recurrent networks; sequence learning; recursion; center embedding; rules. Introduction 0B What kind of structural system underlies human syntactic processing ability? Much work in linguistics addresses this question by exploring syntactic behaviors in natural languages. Work on artificial grammars offers a chance to obtain detailed information about human interaction with formal syntactic systems in the absence of semantic content or task-independent pragmatic function. Here, we introduce a variant on existing artificial grammar learning tasks that supports careful comparison between human and artificial neural network models. The results help clarify the difference between standard, rule-based conceptions of grammatical knowledge and the claims of the neural net perspective, providing some evidence that, at least in the artificial grammar task, humans resemble the networks. We focus, in particular, on the status of center embedding recursion, which many authors view as an important feature of natural language systems, but whose status in the theory of representation has been much debated (e.g., Chomsky, 1957; Christiansen & Chater, 1999; Friederici, 2002). Center-embedding recursive patterning can be generated by context free grammars. Context free grammars are rule systems like Grammar G (Table 1) in which rules take the form (A X 1 X 2 ... X N , for N a finite number), and there are designated starting rules. The grammar is said to generate a finite sequence of symbols, called a “sentence”, if it is possible to make successive substitutions for symbols on the right hand side of a starting rule until no more substitutions can be made; the resulting right hand side is the generated sentence. Grammar G generates the sentences “1 2 3 4” (a Level 1 sentence), “1 1 2 3 4 2 3 4” (Level 2), “1 1 1 2 3 4 2 3 4 2 3 4” (Level 3), etc. In formal language terminology, a case where the system shifts to a deeper level of embedding (here, 1 after 1)—is called a “push” and a case where it shifts back (2 after 4) is called a “pop”. Keeping track of the syntactic dependencies requires correlating the pops with the pushes. The term “recursion” refers to the situation in which a rule can be invoked an unbounded number of times. “Center embedding recursion” is the case in which the symbol for such a repeatedly used rule occurs in the middle of one of the rules with symbols on either side of it (e.g., in G, “S” occurs with “1” to its left and “2” to its right in the first rule). Center embedding context free grammars are of particular interest because a system for generating or recognizing all and only the sentences produced by a center embedding grammar needs an unbounded memory. It is generally recognized that some degree of center embedding is present in natural languages, for there are many situations where natural languages employ patterns within patterns of the same type—e.g., in relative clauses. This suggests that minds have recursive rule systems at their disposal for keeping track of these patterns. The recursive rule system is appealing as an explanation because it permits efficient description of many cases and predicts the way people exercise their language knowledge in many new combinations of words and phrases (Pinker, 1994). Yet humans have great difficulty processing more than a few levels of embedding in natural language (see Lewis, 1996). Similar findings characterize artificial grammar work on recursion (de Vries, Monaghan, Knecht, & Zwitserlood, 2008; Poletiek, 2002). If a symbol processing system must only handle a few levels of embedding, then it is not strictly necessary to employ a recursive process—a weaker, finite- state device, which has a limited memory capacity, can do the job. Proponents of recursive rules have suggested that memory limitations obscure a fundamentally infinite Table 1: Grammar G. Both rules are starting rules. S → 1 S 2 3 4 S → 1 2 3 4