Adaptive Malicious URL Detection: Learning in the Presence of Concept Drifts
Guolin Tan, Peng Zhang, Qingyun Liu, Xinran Liu, Chunge Zhu, Fenghu Dou · 2018
Hackers can implant malwares (e.g. Trojans, Worms, etc.) in the web pages to steal user information and acquire money illegally. In fact, it is noted that close to one-third of all websites are potentially malicious in nature. Therefore, It makes sense to quickly detect malicious URLs on the Internet. Different from most of previous methods, in this paper, we propose a method for online malicious URL detection based on adaptive learning. By collecting the network traffic from backbone networks, we train machine learning models to detect malicious URLs. But there is a serious problem in dynamically changing environments where the statistical properties of target variable change over time, which is known as concept drift. To address this problem, we apply a nonparametric test to correctly detect concept drifts in adaptive learning. Extensive experiments with different types of concept drifts are performed to demonstrate the feasibility of our proposed method on both artificial and real datasets. Our empirical study shows that this approach has good performance in detecting malicious URLs and concept drifts.