PSYCHOMETRIC AND COGNITIVE ANALYSIS AS A BASIS FOR THE DESIGN AND REVISION OF QUANTITATIVE ITEM MODELS
Edith Aurora Graf, Stephen Peterson, Manfred Steffen, René R. Lawless · ETS Research Report Series · 2005
We describe the item modeling development and evaluation process as applied to a quantitative assessment with high-stakes outcomes. In addition to expediting the item-creation process, a model-based approach may reduce pretesting costs, if the difficulty and discrimination of model-generated items may be predicted to a predefined level of accuracy. The development and evaluation of item models represents a collaborative effort among content specialists, statisticians, and cognitive scientists. A cycle for developing and revising item models that generate items with more predictable statistics is described. We review the goals of item modeling from different perspectives and recommend a method for structuring families of models that span content and generate items with more predictable psychometric parameters.