Predicting malicious hosts by blacklisted IPv4 address density estimation
Dmytro Likhomanov, Vladyslav Poliukh · 2020
Internet resource blacklisting is a well-known and effective technique for mitigation of Internet threats. Blacklisting of Internet Protocol (IP) addresses and domains is present as a de-facto component in a variety of systems such as threat intelligence services, intrusion detection systems (IDS), intrusion prevention systems (IPS). Nevertheless, it suffers from an obvious flaw – inability to mitigate threats from zero-day malicious Internet resources. To address this issue numerous predictive blacklisting approaches were developed in recent years. This study presents a novel predictive blacklisting approach, based on blacklisted IP probability density function estimation with Gaussian Mixture Model (GMM). We evaluate the developed approach on two tasks: estimation of model accuracy in predicting future blacklisted IPs and intersection of generated blacklist with real blacklist database. Experiments provided on the dataset of total 270200 unique blacklisted IPs collected from the list of open blacklist providers. We compare our solution both with recent predictive blacklisting solutions and with rule-based approaches. Experimental results show that proposed GMM-based approach outperforms the recent predictive blacklisting approaches in terms of new threats mitigation: it has 2-4 times smaller "predicted" blacklist, high F1-score of 0,916 (which means low level of false detections). As a result, our approach provides a better solution for realworld threat prevention scenarios. Another contribution of this paper is the evaluation of overall applicability of proximity-based methods for blacklisted IP prediction tasks.