Out-of-Distribution Detection with Confidence Deep Reinforcement Learning
Di Wang · 2023
Effective out-of-distribution (OOD) detection is vital for deep neural networks. The failure to recognize OOD samples from in-distribution (ID) samples can damage the performances of deep neural networks. Confidence-based OOD detection is an essential series of approaches with less computational complexity and easy deployment. However, the confidence estimations are dynamic and uncertain during the training process. Small modifications in inputs can cause neglectable changes in the confidence predictions. These introduced errors in the OOD detector damage the confidence estimation via rejecting OOD samples wrongly. Besides, current confidence-based approaches require domain knowledge to define OOD samples and penalties. This paper proposes a confidence-based deep reinforcement learning OOD detector to bridge these gaps. Without domain knowledge, our proposed algorithm can detect OOD samples with modified confidence values in the uncertain and dynamic training process. Our approach is compared with two OOD datasets, two neural network models, and three baselines to prove its effectiveness. Furthermore, our approach addresses the challenge of input attacks during the training process. Our algorithm is designed to mitigate the impact of such attacks by learning robust policies. According to experiments, our proposed deep reinforcement learning algorithm can surpass the performances of these commonly used baselines.