Automated quantitative analysis of open-ended survey responses for transportation planning
Karl Severin, Swapna S. Gokhale, Karthik C. Konduri · 2017
Many organizations, including transportation agencies, use open-ended questions on their surveys to provide respondents an opportunity to narrate their concerns in their own words. Although these narrative responses contain wealth of useful information, most organizations ignore them because they are complex to interpret and analyze in automated manner. Therefore, this paper proposes an approach to extract useful knowledge from narrative survey responses. Posing the problem as a multi-label classification, the approach uses a Naive Bayes classifier to label open-ended responses as per the topics into which the forced choice responses are grouped. The approach is illustrated using narrative responses to a Customer Metro North Classification Survey. The classifier accuracy ranges from 75%-80% on an average, which is roughly only 10%-20% worse compared to human classifiers. Some categories, however, were harder for the automated classifier to distinguish. These results reveal that automated analysis of survey responses is feasible, but a careful consideration of the survey design into clearer and easily distinguishable categories would improve the accuracy of classification. An analysis of the top ten most commonly occurring words in each class further sheds light into the common concerns voiced by the respondents in each category.