Computerized adaptive testing and equating methods with nonparametric irt models
Jeffrey A. Douglas, Xueli Xu · 2004
Nonparametric item response models have been developed as alternatives to the relatively inflexible parametric item response models. An open question is whether it is possible and practical to conduct equating and administer computerized adaptive testing (CAT) with nonparametric models. This thesis is meant to explore the possibility of these two applications of nonparametric item response models. A central challenge of nonparametric CAT lies in the fact that the derivatives of nonparametric item characteristic curves may not be estimated well, which eliminates the availability of the standard maximum Fisher information criterion. As alternatives, procedures based on Shannon entropy and Kullback-Leibler information are proposed. For a long test, these procedures, which do not require the derivatives of the ICCs, become equivalent to the maximum Fisher information criterion. Two simulation studies are conducted to study the behavior of these two procedures, compared with random item selection. Both studies show that the procedures based on Shannon entropy and Kullback-Leibler information perform similarly in terms of root-mean-square-error, and perform much better than random item selection. An adaptive weighting procedure is proposed to control the item exposure rate without losing much efficiency in latent ability estimation. The simulation studies confirm the expectation. The flexible forms of nonparametric IRT models make test equating more challenging. Though linear equating under parametric IRT models is obvious and appropriate, it might not be appropriate for nonparametric models. Two approaches are proposed for test equating and examined through simulation as well as with real data analysis. The simulation studies show that both approaches are able to recover the true equating functions with tolerable error. The real data analysis shows that these two approaches lead to similar equating functions.