Modeling conditional probabilities in complex educational assessments
Robert J. Mislevy, Russell G. Almond, Louis V. DiBello, Frank J. Jenkins, Linda S. Steinberg, Duanli Yan · PsycEXTRA Dataset · 2002
during June and July. Our subject matter expert consultants were invaluable in working through the issues of standards, claims, and evidence that underlie the project, and in offering suggestions along the way for the prototype. They are Ann Kindfield, Dirk Vanderklein, Scott Kight, Cathryn Rubin, Sue Johnson, and Gordon Mendenhall. For providing data on early field trails of Agouti Segment 1, we thank the ETS Summer 2000 Interns, the Weston scholars at Montclair State University and their advisor Prof. Lynn English, and Russell’s buddies at the Knight Dreams comic book shop. Modeling Conditional Probabilities in Complex Educational Assessments An active area in psychometric research is coordinated task design and statistical analysis built around cognitive models. Compared with classical test theory and item response theory, there is often less information from observed data about the measurement-model parameters. On the other hand, there is more information from the grounding psychological theory, and the task-designer’s insights into which patterns of skills lead to which patterns of performance. We describe a Bayesian approach to modeling these situations, which uses expert ’ judgments to produce prior distributions for