A scalable network event detection framework for darknet traffic
Max Gao, Ricky K. P. Mok · 2022
Unsolicited network traffic captured by network telescopes, namely darknet traffic, provides important data for studying malicious Internet activities, such as network scanning [9], the spread of malware [4], and DDoS attacks [6]. Inferring such activity in traffic often requires first obtaining fingerprints of the activity and searching historical traffic traces (e.g, pcaps) for that pattern. Traffic volume at the largest darknets can exceed 100GB/hour, rendering it challenging to process at the packet level. Aggregated flow-based metadata [2] can reduce computation, storage and I/O overhead at the expense of finer-grained information about the traffic. Customized data structures (e.g., [7]) and streaming algorithms (e.g., [5]) offer an alternative approach to extracting information from raw packets, but they are typically traffic tailored for estimating specific metrics and thus limited in their ability to detect a wide range of events.