Application of Sequential Methods to Testlet Based Educational Test- ings
Yuan-chin Ivan · 2011
where Yij denotes the response of a test-taker with the ability i to the item j, parameters aj, bj and cj retain their original interpretation as in the item responser theory, and the newly introduced parameter ri;k(j) is the parameter of testlet effect of item j with person i that is nested within testlet k. Note that if ri;k(j) = 0 for all i, j and k, then (1) becomes the classical 3-parameter logistic model used in the item response theory. Suppose those item parameters are known in advance, and our goal is to estimate the ability levels, ’s, of test takers. Then for this kind of the correlated binary responses data, the method of the generalized estimating equation (GEE) (Liang and Zeger, 1986) can be used. In addition, as in the variable length computerized adaptive testing, it is of interest to know how many testlets used will suffice to obtain an estimate of with a satisfactory (prescribed) accuracy. In this study, we use a fixed width confidence interval to manage the accuracy of estimation of , and a sequential method is employed such that the test is stopped as long as the prescribed accuracy for the estimate is reached. Assume the exchangeable correlations among items within a testlet, and items from different testlets are mutually independent. Thus, GEE with a “diagonal block” working covariance matrix is used. For a given test-taker i, let corr(Yik(j);Yik(j′)) = k, j = j ′ , be the correlation between responses Yik(j);Yik(j′) to items j;j ′ , respectively, in the k-th testlet. Then correlation matrix for observations