Incivility or Invalidity? Evaluating Perspective API Scores as a Measure of Political Incivility
Bryan T. Gervais, Connor Dye, Amber Chin · American Politics Research · 2025
Adopting toxicity scores produced by Google’s Perspective API has become perhaps the most popular approach to estimating the prevalence of incivility and toxicity in online communication. However, existing scholarship has often overlooked the imbalanced nature of incivility when evaluating the performance of Perspective scores, leading to an overestimation of their effectiveness. In this research note, we demonstrate that once this imbalance is considered, Perspective scores perform poorly in accurately identifying incivility. We then propose a method that leverages Perspective scores to reduce the manual coding needed for developing a high-performing supervised learning classification model. Our findings indicate that combining Perspective scores with supervised learning approaches yields significantly better results for identifying incivility.