Developing and Validating Cognitive Models in Assessment
Madeleine Keehner, Joanna S. Gorin, Gary Feng, Irvin R. Katz · 2016
In this chapter, we first attempt to characterize cognitive models in ways that are useful to the assessment domain and provide a rationale for developing and validating cognitive models for the applied goals in this field. Having clarified the value of cognitive models for assessment, we introduce different forms of evidence that have promise and practical use for validating cognitive models in this domain. We divide these into off-line evidence (which can be collected in studies devised and implemented after or before the assessment task has been administered) and online evidence (which is gathered directly from test-takers or research participants while they are completing assessment tasks). Finally, we describe an instance where we combined multiple forms of evidence in the development phases of a large-scale testing program, in order to illustrate how different evidence types can be collected and combined during assessment development and analyzed for insights into student cognition.