The Explainability–Reliability Gap in Fraud Detection: Evidence from SHAP and Permutation Importance Under Distribution Shift
Istiaque Bhuiyan, Rahma Mirza, Ariful Hoque, Tanvir Bhuiyan · FinTech · 2026
This study develops an empirical audit framework for assessing the explainability–reliability gap in fraud detection: whether stable model explanations remain consistent with performance-based feature reliance under distribution shift. Using the Bank Account Fraud dataset suite, including a Base dataset and five biased variants, the study examines group-size disparity, fraud-prevalence disparity, separability bias, and temporal shift. Logistic Regression, Linear SVC, and Random Forest are benchmarked using standard classification metrics, followed by cross-variant evaluation with Logistic Regression as the interpretable baseline. SHAP is used to assess explanation stability, while permutation importance measures performance-based feature reliance. The results show that accuracy and ROC-AUC can overstate practical effectiveness under severe class imbalance; notably, Random Forest retained useful discrimination while producing near-zero recall at the evaluated threshold. SHAP feature rankings remained relatively stable across variants, particularly for address-history, identity-similarity, credit-risk, device, and behavioral variables. However, permutation importance revealed weaker and more variable reliance on several SHAP-ranked features. The limited agreement between the two measures indicates a partial explainability–reliability gap. The findings show that explanation stability alone is insufficient for evaluating trustworthy fraud detection models and should be complemented by performance-based validation under biased and shifted deployment conditions.