A Novel Anomaly Detection Framework for Big Network Traffic Data
Hailong Li, Honggang Wang, Xiao Peng, Ge Zhang · 2024
Monitoring network traffic to identify malicious applications is an active research topic in network security. In the era of big data, with the increasing number of access devices, networks will become more and more dense and require more rapid response. Traditional anomaly detection methods cannot achieve both detection accuracy and latency. To study a high-performance network traffic anomaly detection method is imperative. In this paper, We proposed a novel anomaly detection framework based on big data analytics for network traffic to enhance the detection of sophisticated cyber threats. The framework is divided into two stages: online and offline. The online stage uses a distributed algorithm utilizing customized network traffic characteristics. In the offline stage, the multi-modal method of accurate classification is adopted, and the identification result is used as an expert system of online algorithm to realize data authentication. By integrating advanced machine learning algorithms and big data parallel anomaly detection techniques, the proposed framework aims to strike a balance between accuracy and efficiency in detecting emerging cyber threats. The framework has been tested on both open datasets and real-world datasets. A large number of experiments have been conducted to validate the feasibility and practicality of the framework.