Web crawler design, implementation and data analysis about Sina Weibo
Wang Dawei, Muzaffar Hamzah, Giap Weng Ng · 2019 3rd International Conference on Electronic Information Technology and Computer Engineering (EITCE) · 2019
In this paper, discussed designed and implemented a crawler base on python language and use this program crawl posts from Sina weibo. The key word is academic library. The time span is from 1/1/2018 to 31/12/2018 and the total number is 39374 and make data clean, cut words and delete all text that no related to academic library. After that, analyzed this data set for knowing the user’s attitude about library. In order to describe the situation better, high frequency words rank, cloud word, semantic network about image of Library service, semantic network been used. Finally, according to the user's attitude, make the the necessary discuss.