Intrusion Anomaly Detection Based on Pseudo-Count Exploration
Feifei Cao · Research Square · 2023
Abstract With the increase in cyber-attacks, the issue of risk to web services is of great concern, making it particularly important to apply new techniques to improve the performance of intrusion detection systems. This study presents a novel application of a deep reinforcement learning algorithm based on pseudo-count exploration for network intrusion anomaly detection. We describe how to introduce a pseudo-count exploration (PCE) approach for supervised intrusion anomaly detection based on deep reinforcement learning (DRL) algorithms and construct an exploration space based on their architecture. For intrusion detection systems (IDSs), setting up dense extrinsic rewards consistent with intrusion detection is very difficult because the identifications are all manual and cannot be automatically identified and these identified intrusion events are stored to form a network feature dataset. We select the NSL-KDD and AWID datasets and compare our proposed new models based on the pseudo-count exploration of double deep Q networks (PCE-DDQN) and Proximal Policy Optimization (PCE-PPO) with existing machine learning and traditional DRL techniques. Our models significantly outperform the other two types of models in terms of detection performance, especially on the AWID dataset where the anomaly distribution is highly unbalanced. In addition, in terms of training cost, our model is faster than the traditional DRL model. We also discuss the effect of discount factor size on the detection ability of the models, and our models achieve excellent performance on all three types of metrics.