Item Development and Pretesting in a CBT Environment
Craig N. Mills, Maria T. Potenza, John J. Fremer, William C. Ward · 2005
The test development process typically requires that many more items be written then will be used, even across multiple forms. Once items have been written, they must be evaluated for conformance to certain format and style considerations. More critically, they are evaluated in terms of testing program requirements for levels of content, difficulty, and fairness. Items that satisfy initial reviews are pretested, or administered to program examinees. Results of the pretesting are used to inform decisions about these items. Poorly performing items are removed from further consideration; items that demonstrate less serious problems are modified (followed by another round of pretesting); items that demonstrate still better performance are incorporated into the test program bank or pool. Classical item statistics may be used to ensure items in the bank satisfy certain criteria for difficulty, discrimination, and distractor performance. IRT criteria for the item statistics may specify acceptable ranges or values for the b, a, and c item parameters.