"Hey #311, Come Clean My Street!": A Spatio-temporal Sentiment Analysis of Twitter Data and 311 Civil Complaints
Ryan Eshleman, Hui Qiong Yang · 2014
Twitter data has been applied to address a wide range of applications (e.g., Political election prediction and disease tracking), however, no studies have been conducted to explore the interactions and potential relationships between twitter data and social events available from government entities. In this paper, we introduce a novel approach to investigate the spatio-temporal relationships between the sentiment aspects of tweets and 311 civil complaints recorded in the 311 Case Database, which is freely available from the City of San Francisco. We also present results from two supporting tasks: (1) We apply sentiment analysis techniques to model the emotional characteristics of five metropolitan areas around the globe, allowing one to gain insight into the relative happiness across cities and neighborhoods within a city, and (2) we quantify the performance of several open-source machine learning algorithms for sentiment analysis by applying them to large volume of twitter data, thereby providing empirical guidelines for practitioners. Major contributions and findings include (1) We have developed a system for the relative ranking of happiness of a geographical area. Our results show that Sydney, Australia is the happiest of the five cities under study, (2) We have found a counterintuitive positive correlation between 311-report frequency and local sentiment, and (3) When performing sentiment analysis of tweets, the inclusion of emoticons in the training dataset can lead to model over fitting, whereas NLP-based features seem to have a great potential to improve the classification accuracy.