Evaluation of Anomaly-Based Network Intrusion Detection Systems with Unclean Training Data for Low-Rate Attack Detection
Angela O. Prabowo, Deka Julian Arrizki, Baskoro Adi Pratomo, Ahmad Ibnu Fajar, Krisna Badru Wijaya, Hudan Studiawan, Ary Mazharuddin Shiddiqi, Siti Hajar Othman · Journal of Cybersecurity and Privacy · 2026
Anomaly-based network intrusion detection systems (NIDSs) complement signature-based detection methods to identify unknown (zero-day) attacks. The integration of machine and deep learning enhanced the efficiency of such NIDSs. However, since anomaly-based NIDSs heavily depend on the quality of the training data, the presence of malicious traffic in the training set can significantly degrade the model’s performance. Purging the training data of such traffic is often impractical. This study investigates performance degradation caused by increasing amounts of malicious traffic in the training data. We introduced varying portions of malicious traffic into the training sets of machine and deep learning models to determine which approach is most resilient to unclean training data. Our experiments revealed that Autoencoders, using a byte frequency feature set, achieved the highest F2 score (0.8989), with only a minor decrease of 0.0009 when trained on the most contaminated dataset. This performance drop was the smallest compared to other algorithms tested, including an Isolation Forest, a Local Outlier Factor, a One-Class Support Vector Machine, and Long Short-Term Memory.