Towards Trustworthy AI: Real-Time Uncertainty Monitoring and Adversarial Detection
Chern Chao Tai, Abhijeet Solanki, Wesam Al Amiri, Douglas A. Talbert, Syed Rafay Hasan, Terry N. Guo · 2025
Autonomous vehicles (AVs) rely heavily on perception sensors and artificial intelligence (AI) to perform critical navigational perception tasks, such as object detection, distance measurement, and classification. These tasks depend on data from sensors like cameras, LiDAR, and radar, which are processed by deep learning models. However, existing systems often assume that AI models are inherently trustworthy, overlooking their vulnerability to noise or false data injection attacks that can degrade performance and lead to misclassifications. To address this challenge, we propose a novel two-stage framework designed to quantify uncertainty in AI predictions of data captured from camera sensors and enable continuous monitoring, which is a foundational step toward implementing Zero Trust (ZT) principles in AV systems. Our approach integrates Monte Carlo (MC) dropout in deep learning architectures (e.g., VGG16) with One-Class Classification methods to detect anomalous or adversarial inputs. Additionally, we introduce a sliding window batch monitoring technique capable of capturing temporal patterns of adversarial attacks, thereby enhancing robustness through contextual analysis. To address computational overhead challenges, we introduce a layer-reutilization based optimization technique for Monte Carlo dropout that achieves 2.42-3.58× speedup while preserving uncertainty quantification quality. Extensive experiments across varying perturbation levels and MC sample sizes demonstrate that our single-image detection framework and batch monitoring approach both achieve strong anomaly detection performance, with the latter particularly effective in identifying temporally clustered adversarial inputs.