Comparing Uncivil Discourse with a Conversational Intelligent Agent in Public and Private Online Settings
Mark Marcello · RIT Scholar Works (Rochester Institute of Technology) · 2015
According to a recent study by the Pew Research Center, 60% of participants of a survey said they have witnessed online abuse, and 66% of that abuse occurred via social media. This study looks at online communication, and tests if anonymity increases the amount of abuse that occurs during online communication. A quasi-experiment was developed to look for uncivil discourse using a conversational intelligent agent to create a controlled, repeatable conversational environment. The interactions with the agent occur in two settings: one in public at a kiosk, and a second in private. The researcher hypothesized that the interactions that occur in private would be more abusive than the interactions that occurred in private, due to the anonymity of the conversations. Several thousand interactions were recorded in the two scenarios, and analysis was performed on the conversational data. A custom search engine used a data set of known abuse keywords to flag each conversation as abusive or not abusive. Using this abusive conversation count, a chi-square test of independence was used to determine that the number of abusive interactions with the agent was statistically higher in the private scenario compared to the public scenario. This finding supported the researcher’s hypothesis, showing that users were almost twice as likely to use abusive keywords when interacting with the agent in a private setting.