From prediction to trust: enhancing deep learning models for insurance fraud detection through uncertainty quantification
Ziad Salem, Samer Sahl, Mohamed Hanafy Kotb Ibrahim · PeerJ Computer Science · 2025
This study applies uncertainty quantification techniques to deep neural networks for automobile insurance fraud detection to address the critical need for confidence estimation in artificial intelligence (AI)-driven decision systems. We evaluate three uncertainty quantification approaches: Monte Carlo Dropout (MCD), Deep Ensembles, and Ensemble Monte Carlo Dropout (EMCD) on a dataset of 15,420 insurance claims. Our framework incorporates SHapley Additive exPlanations (SHAP) based interpretability analysis and introduces uncertainty-specific evaluation metrics through an uncertainty confusion matrix. We examine how data resampling techniques (Random Over-Sampling (ROS) and Synthetic Minority Oversampling Technique (SMOTE)) affect both predictive performance and uncertainty calibration. Results demonstrate that while resampling improves fraud detection sensitivity, it increases predictive uncertainty by 65.12% to 298.55%. Critically, this increased uncertainty strongly correlates with model misclassifications, indicating improved self-awareness of prediction reliability. The EMCD approach achieves the highest fraud detection rates but with elevated uncertainty, while ensemble methods provide more conservative predictions with better calibration. These findings contribute to developing trustworthy AI systems for insurance fraud detection by providing both predictions and associated confidence measures.