Anomalous Host Identification in High Cardinality Network Streams using Adaptive Reverse Sketching
Taimur Bakhshi, Bogdan Ghita, Abiodun Brimmo Yusuf · 2024
Network-connected hosts reporting high cardinality changes may exhibit anomalous behaviour and remain an evolving cyber security risk for the network infrastructure. High-speed line-rate detection of cardinality changes by network monitors presents a significant challenge for security and network operations center (SoC/NoC) analysts to accurately identify anomalous hosts and mitigate threats in real time. State-of-the-art monitoring solutions have typically focused on specific functionalities limited to host address detection, overhead cost optimization, and distributed cardinality measurements. However, real-time anomaly detection may benefit from a holistic operational approach. In this study, we propose the use of an adaptive reverse sketching scheme for anomaly detection in high-cardinality traffic streams using a combination of machine learning threshold prediction and reverse sketching from traffic (data structure) derivation. The scheme offers a parallel distinction between source and destination hosts based on adaptive counters, accounting for minor and large cardinality changes among hosts while optimizing the resource consumption. During the validation phase, the proposed scheme was evaluated and compared with existing approaches on locally generated and CAIDA 2007 datasets. The present model reported an accuracy improvement by a margin of 32-46% when compared with previous approaches in cardinality change estimation and abnormal host address identification.