Modeling Individual Differences in Category Learning - eScholarship
Michael R. Webb, Michael Lee · Proceedings of the Annual Meeting of the Cognitive Science Society · 2004
Modeling Individual Differences in Category Learning Michael R. Webb ([email protected]) Command and Control Division, Defence Science and Technology Organisation Edinburgh, South Australia, 5111, AUSTRALIA Michael D. Lee ([email protected]) Department of Psychology, University of Adelaide South Australia, 5005, AUSTRALIA Abstract Many evaluations of cognitive models rely on data that have been averaged or aggregated across all ex- perimental subjects, and so fail to consider the possi- bility that there are important individual differences between subjects. Other evaluations are done at the single-subject level, and so fail to benefit from the reduction of noise that data averaging or aggrega- tion potentially provides. To overcome these weak- nesses, we develop a general approach to modeling individual differences using families of cognitive mod- els, where different groups of subjects are identified as having different psychological behavior. Separate models with separate parameterizations are applied to each group of subjects, and Bayesian model selec- tion is used to determine the appropriate number of groups. We demonstrate the general approach in a concrete and detailed way using the ALCOVE model of category learning and data from four previously analysed category learning experiments. Meaningful individual differences are found for three of the four experiments, and ALCOVE is able to account for this variation through psychologically interpretable differ- ences in parameterization. The results highlight the potential of extending cognitive models to consider in- dividual differences. Introduction Much of cognitive psychology, as with other empiri- cal sciences, involves the development and evaluation of models. Models provide formal accounts of the ex- planations proposed by theories, and have been de- veloped to address diverse cognitive phenomena rang- ing from stimulus representation (e.g., Shepard 1980), to memory retention (e.g., Anderson & Schooler 1991; Estes 1997), to category learning (e.g., Ashby & Per- rin 1988; Berretty, Todd, & Martignon 1999; Kruschke 1992; Tenenbaum 1999). One recurrent shortcoming of these models, however, is that (whether intentionally, or as an unintended consequence of methodology) hu- mans are usually modeled as ‘invariants’, and not as ‘individuals’. This occurs because, most often, mod- els are evaluated against data that have been averaged or aggregated across subjects, and so the modeling as- sumes that there are no individual differences between subjects. The potential benefit of averaging data is that, if the performance of subjects really is the same except for ‘noise’ (i.e., variation the model is not attempting to explain), the averaging process will tend to remove the noise, and the resultant data will more accurately re- flect the underlying psychological phenomenon. When the performance of subjects has genuine differences, however, it is well known (e.g., Estes 1956; Myung, Kim, & Pitt 2000) that averaging produces data that do not accurately represent the behavior of individuals, and provide a misleading basis for modeling. Even more fundamentally, the practice of averaging data restricts the focus of cognitive modeling to issues of how people are the same. While modeling invariants is fundamental, it is also important to ask how people are different. Experimental data reveal individual dif- ferences in cognitive processes, and in the psychological variables that control those processes, that also need to be modeled. Cognitive modeling that attempts to accommodate individual differences usually assumes that each sub- ject behaves in accordance with a different parame- terization of the same basic model, and so the model is evaluated against the data from each subject sep- arately (e.g, Ashby, Maddox, & Lee 1994; Nosofsky 1986; Wixted & Ebbesen 1997). Although this avoids the problem of corrupting the underlying pattern of the data, it also foregoes the potential benefits of aver- aging, and guarantees that models are fit to all of the noise in the data. Another problem with individual subject analysis, from a model theoretic perspective, is that fitting each additional subject requires an extra set of free parame- ters, and so leads to a progressively more complicated accounts of the data as a whole. As has been pointed out repeatedly in the psychological literature recently (e.g., Myung & Pitt 1997; Pitt, Myung, & Zhang 2002), it is important both to maximize goodness-of- fit and minimize model complexity to achieve the basic goals of modeling. Unnecessarily complicated models that “over-fit” data often do not provide any insight or explanation of the cognitive processes they address, and are less capable of making accurate predictions when generalizing to new or different situations.