Detecting Zero-Day Attacks using Advanced Anomaly Detection in Network Traffic
Asma Anjum, P. Rama Subramanian, R. Stalinbabu, K. Deepthi, K. Santha Sheela, B. Jegajothi · 2025
Zero-day attacks pose a significant challenge to cybersecurity due to their unpredictable nature and the lack of labeled attack data for training conventional detection models. This research proposes a Hybrid Deep Anomaly Detection Model that integrates Autoencoders, Transformer-Based Detection, and Isolation Forest to enhance the identification of zero-day attacks in network traffic. The Autoencoder component learns normal traffic patterns and identifies deviations based on reconstruction errors. A Transformer-based model captures temporal dependencies in network traffic using self-attention mechanisms, improving the representation of evolving attack behaviors. Finally, an Isolation Forest refines anomaly detection by isolating outliers and reducing false positives. The model is trained and evaluated on the CSE-CIC-IDS2018 dataset, which contains diverse cyberattack scenarios, ensuring a robust performance assessment. The results of the proposed model achieve 97.2% accuracy and a false positive rate of 2.8%, significantly outperforming existing approaches such as Autoencoder-only, LSTM-based detection, and Random Forest classifiers. The proposed framework enhances zero-day attack detection, minimizes false alerts, and ensures real-time adaptability in modern cybersecurity environments.