A Social Media-Machine Learning Approach to Detect Public Perception of Transportation Systems
Qifan Nie, Jake Zarobsky, Pezhman Sheinidashtegol, Aibek Musaev, Andrew J. Graettinger, Weike Lu, Guojing Hu · 2023
Social sensors such as Facebook and Twitter are used by the public to express their perceptions and circumstances in various aspects of their lives. People publish tweets and post Facebook messages related to transportation systems to report what they have seen on the road, especially when an unusual or unsafe situation (e.g., pothole, traffic crash) occurs. Some messages directly refer to a public agency’s account (e.g., State Department of Transportation), expecting the public agency to respond to these posts. Many posts contain real-time urgent incident information (e.g., broken traffic lights, flooding) or other inquiries, comments, requests, suggestions, and complaints about transportation systems. However, many of such posts fail to get a timely response from related agencies, perhaps because public agencies do not have an efficient way to identify the public’s perception of the transportation system in their jurisdiction. The objective of this study was to develop a novel approach to efficiently monitor and detect critical real-time information from public awareness. This study used tweets for demonstration. A keyword-extraction process using the Term Frequency method was implemented to gather valid transportation terms in public tweets. A machine learning-based model was trained to identify transportation-related tweets from the live stream. The model training started with manually identifying relevant tweets (historical data). The results showed that, for identifying transportation-related posts, the trained machine learning model’s overall performance was 0.848. Furthermore, this trained model has been applied in a live streaming data collection process, and 81.86% of the total collected tweets were manually identified as relevant tweets. Hence, this framework can be employed by public agencies to identify important public posts related to transportation in a timely and proactive manner.