Exploring the effectiveness of intersectional routing for multistage testing with mixed-format tests
Korean Society for Educational Evaluation, Hyesung Shin, Guemin Lee · 교육평가연구 · 2024
Multistage testing among several adaptive testing types has been more frequently applied into practice because it can accurately and efficiently estimate abilities with relatively short tests. However, it has been also criticized due to its low level of adaptability. To address this issue, the intersectional routing method was proposed and its effectiveness has been partly confirmed. This study aims to explore the effectiveness of the intersectional routing method with mixed-format MST by applying the bi-factor MIRT model, and the UIRT model, extending previous researches. Based upon the results of this study, when the module length was short (8 items), and the correlation between sections was high (0.9), the accuracy of ability estimates using the intersectional routing was higher than the typical multistage testing design regardless of the IRT model. The application of the intersectional routing technique decreased the differences of ability estimate accuracy in accordance with panel designs. Considering the high inter-correlation values between sections in actual tests (ranging from 0.6 to 0.9), the application of the intersectional routing method into multistage testing could be a choice for broadening options in test designs and obtaining more accurate ability estimates. The results also indicate the necessity of selecting an appropriate IRT model based on test characteristics in multidimensional adaptive testing situations.