Detecting Differential Item Functioning (DIF) in Multidimensional Item Response Theory (MIRT) Models Using Explainable Artificial Intelligence (XAI)
Karoline Wolandt, Elisabeth Barbara Kraus · Psychological Test Adaptation and Development · 2026
Abstract: We conducted two studies on using random forests (RFs) with explainable artificial intelligence (XAI) to detect differential item functioning (DIF) in multidimensional item response theory (MIRT) models. RF-XAI identifies DIF-items by their importance in predicting group membership from item responses and person parameters. Study 1 examines the impact of test characteristics, specifically DIF-item proportion, test length, sample size, test dimensionality, and the RF parameter mtry on variable importance and detection metrics. High detection rates and low false-positive rates occurred in large samples, with low DIF proportions, and medium mtry values. Study 2 compares RF-XAI to Mantel–Haenszel (MH) and logistic regression (LR). RF-XAI slightly outperformed traditional methods in large, multidimensional tests, while MH and LR were more effective in smaller samples and unidimensional scales. The results support RF-XAI as a promising tool for enhancing fairness in psychological and educational assessments.