Research on Dark Web Monitoring Crawler Based on TOR
Yingying Xu, Guangxuan Chen, Jiajian Wu, Wanpeng Xu, Qiang Liu · 2021 IEEE 2nd International Conference on Information Technology, Big Data and Artificial Intelligence (ICIBA) · 2021
The emergence of anonymity services provides the role of protecting user information security, it also provides a perfect venue for illegal and criminal activities such as human trafficking, illegal information transactions, and drugs and firearms. The anonymity, complexity of the structure, high-efficiency information distribution, and weak legal constraints of the dark web make it a new carrier for many criminals to carry out criminal activities. Therefore, in order to supervise criminals using the “dark web” to conduct illegal activities, a Python dark web monitoring crawler was designed based on Tor to obtain and store a large number of dark web site addresses; a web crawler based on the scrapy framework was established to crawl the dark web and specific types of crimes. Relevant website information is saved in the MongoDB database; data analysis is carried out by designing a crawler algorithm; finally, the analyzed data is generated through intuitive word cloud diagrams, histograms and other methods to generate a visual interface to facilitate real-time monitoring of dark web crimes.