A Reinforcement Learning-based Filter for Adaptive Anomaly Detection in Few-Shot Scenarios
Yazhuo Gao, Yang Lin, Ran Zhu, Yixuan Wu, Yining Cao · 2025
With the rapid advancement of computer network technologies, the complexity of cybersecurity issues has grown significantly. Anomaly detection as a critical role of defense against cyberattacks, play an essential role in safeguarding network security. However, traditional Anomaly detection methods often struggle to address few-shot scenarios. To overcome this limitation, an adaptive anomaly detection method for few-shot scenarios, RL-FAD (Reinforcement Learning-based Filter for Anomaly Detection), has been proposed. This method leverages causal feature selection and reinforcement learning to enhance detection performance. RL-FAD begins by efficiently and accurately identifying an optimal subset of features that predict target variables through a causality-based conditional test and a Markov blanket search algorithm. It then utilizes a tree model to transform the selected feature vectors, constructing a reinforcement learning environment that adheres to the Markov decision process. A Deep Q-Network (DQN) algorithm is subsequently employed to handle classification decisions. Experimental results demonstrate that RL-FAD can dynamically adjust its classification decisions in response to evolving attack patterns. The method was evaluated on three datasets—CIC-IDS2017, CIC-DDoS2019, and CIC-UNSW-NB15—achieving exceptional performance. Notably, even with only 100 training samples, RL-FAD achieved anomaly detection accuracies of 95%, 96%, and 94% on test datasets containing 50,000 samples, respectively. These results underscore RL-FAD’s robust detection capabilities and significant practical value in enhancing network security.