Differential Item Functioning for Polytomous Response Items Using Hierarchical Generalized Linear Model
Hua Meng · D-Scholarship@Pitt (University of Pittsburgh) · 2019
Hierarchical generalized linear model (HGLM) as a differential item functioning (DIF) detection method is a relatively new approach and has several advantages; such as handling extreme response patterns like perfect or all-missed scores and adding covariates and levels to simultaneously identify the sources and consequences of DIF. Several studies examined the performance of using HGLM in DIF assessment for dichotomous items, but only a few exist for polytomous items. This study examined the DIF-free-then-DIF strategy to select DIF-free anchor items and the performance of HGLM in DIF assessment for polytomous items. This study extends the work of Williams and Beretevas (2006) by adopting the constant anchor item method as the model identification method for HGLM, and examining the performance of DIF evaluation with the presence of latent trait differences between the focal and reference group. In addition, the study extends the work of Chen, Chen, and Shih (2014) by exploring the performance of HGLM for polytomous response items with 3 response categories, and comparing the results to logistic regression and Generalized Mantel-Haensel (GMH) procedure. In this study, the accuracy of using iterative HGLM with DIF-free-then-DIF strategy to select DIF-free items as anchor was examined first. Then, HGLM with 1-item anchor and 4-item anchor were fitted to the data, as well as the logistic regression and GMH. The Type I error and power rates were computed for all the 4 methods. The results showed that compared to dichotomous items, the accuracy rate of HGLM methods in selecting DIF-free item was generally lower for polytomous items. The HGLM with 1-item and 4-item anchor methods showed decent control of Type I error rate, while the logistic regression and GMH showed considerably inflated Type I error. In terms of power, HGLM with 4-item anchor method outperformed the 1-item anchor method. The logistic regression behaved similarly to HGLM with 1-item anchor. The GMH was generally more powerful, especially under small sample size conditions. However, this may be a result of its inflated Type I error. Recommendations were made for applied researchers in selecting among HGLM, logistic regression, and GMH for DIF assessment of polytomous items.