A hash-based algorithm for measuring cardinality distribution in network traffic
Weijiang Liu, Chao Liu, Shuming Guo · International Journal of Autonomous and Adaptive Communications Systems · 2016
The host cardinality, defined as the number of distinct peers that a host communicates with, is an important metric for profiling hosts. Host cardinality distribution is very useful for characterising the communication connectivity patterns between hosts inside a network. With the development of the internet, network intrusion events occur frequently, such as worm propagation, DDoS attacks, port scanning, etc. These attacks generate a lot of traffic connections in a short time, resulting in network block and even paralysis. In the case of DDoS or worm attacks, the infected host usually produces a lot of connections with other hosts in a short period of time, then the host cardinality distribution will be different from normal situations. Hence, this paper proposes a hash-based algorithm for measuring the host cardinality distribution. Combining with hash, Bloom filter, and data stream algorithm, the space and time consumption of the algorithm is very small, so it can be used to estimate the host cardinality distribution in the high-speed network.