Interactions Between Frequenct Effects and Age of Acquisition Effects in a Connectionist Network
Paul Munro, Garrison W. Cottrell · eScholarship (California Digital Library) · 2001
Interactions betw een Frequency Effects and A ge of A cquisition Effects in a C onnectionist N etw ork Paul W . M unro (m unro@ sis.pitt.edu) School of Inform ation Sciences U niversity of Pittsburgh Pittsburgh, PA 15260 U SA G arrison C ottrell (gary@ cs.ucsd.edu) D epartm ent of Com puter Science and Engineering 0114 U niversity of California, San D iego La Jolla, CA 92093-0114 U SA A bstract The perform ance of a connectionist netw ork, in w hich som e resources are absent or dam aged is exam ined as a function of various learning param eters. A learning environm ent is created by generating a set of random “prototypes” and clusters of exem plar vectors surrounding each prototype. A n autoencoder is trained on the patterns. The robustness of each learned item is m easured as a function of the tim e at w hich it w as “acquired” by the netw ork and its overall frequency in the environm ent. Both factors are show n to influence robustness under several learning conditions. A nderson (2001). H ere, w e look at pattern perform ance in the face of dam age to the netw ork, sim ulating neuronal failure as could occur w ith aging or traum a. The robustness of netw ork perform ance to hidden unit dam age has been show n to im prove for netw orks trained w ith noise am ong the hidden units (Judd & M unro, 1993). In som e cases, this kind of noise has been show n to im prove the generalization properties of a netw ork (Clay & Sequin, 1990). Functionally, the hidden representations of the training item s settle to states that are further apart in term s of a Euclidean m easure. Introduction For all their shortcom ings, feed-forw ard netw ork m odels of learning and m em ory share certain im portant features w ith their biological counterparts. A m ong these are the ability to gradually abstract statistical regularities from their environm ents by incorporating them into their connectivity structures and the feature generally know n as “graceful degradation”. In this paper, the relationship betw een early learning (acquisition) and degradation of perform ance through loss of resources is exam ined in the context of sm all- scale sim ulations, in term s of frequency effects, age of acquisition (A oA ) effects, prototype effects, and the insertion of noise into the neural netw ork. The relative influence of A oA com pared to frequency on w ord nam ing tasks has been argued am ong cognitive psychologists and linguists for several years now (Brow n & W atson, 1987; M orrison et al., 1992; G erhard & Barry, 1998). O f course, teasing apart the influences of A oA and frequency is confounded by the strong correlation betw een them . A oA effects have also been reported in other dom ains, such as object identification and face recognition (M oore & V alentine, 1999). The effects of A oA and frequency on pattern error have been analyzed by Sm ith, Cottrell, and In this paper, w e exam ine the follow ing three hypotheses: 1. The robustness of an item under loss of netw ork com putational resources (analogous to the loss of neurons in hum ans) is related both to the tim e at w hich that item w as “acquired”, and to the average frequency of the item in the netw ork’s experience. 2. Prototypical item s are m ore robust than exem plars, even if they are never explicitly presented to the netw ork, since they share features w ith populations of exem plars, and thus have high “effective frequencies” in the environm ent. 3. Early explicit learning of prototypes can result in a m ore robust set of internal exem plar representa- tions. M ethodology The training set A tw o-step process is used to generate a structured set of bit strings of length L. First, a set of N prototype strings is produced by generating 0 and 1 values