The Application of Web Crawler in City Image Research
Jiangying Xu, Lixin Du, Chunsun Duan, Mingyue Li · 2020
Grasping the city image accurately is always a challenge for city manager. The main work of our study is to capture the web pages of a specific city using web crawler technology, process and analyze the captured data to get keywords representing the city. We propose a distributed crawler architecture and collect city-related data based on the architecture. Subsequently, several example specific cities are crawled successfully. The crawled data are analyzed by using data preprocessing, data analysis, and other methods. The results show that the presented approach will be very useful for city manager.