Automatic Smart Crawling on Twitter for Weather Information in Indonesia
Kartika Purwandari, Reza Bayu Perdana, Join W. C. Sigalingging, Reza Rahutomo, Bens Pardamean · Procedia Computer Science · 2023
Although Twitter is a popular platform for social interaction analysis and text data mining, it faces challenges with geolocation automation. To address this problem, the researchers propose the utilization of a Support Vector Machine (SVM) model to develop an automated Twitter crawling system. The system aims to collect data related to weather in Indonesia by employing Twin, a Python-based Twitter scraping software. To overcome null geolocations, the study incorporates aliases created based on the common practice of Indonesian users mentioning the country's location in tweets. The results demonstrate that the SVM model, combined with automated smart crawling, achieves an 85% accuracy rate.