OLEA: Tool and Infrastructure for Offensive Language Error Analysis in English
Marie Grace, Jay Seabrum, Dananjay Srinivas, Alexis Palmer · 2023
State-of-the-art models for identifying offensive language often fail to generalize over more nuanced or implicit cases of offensive and hateful language.Understanding model performance on complex cases is key for building robust models that are effective in real-world settings.To help researchers efficiently evaluate their models, we introduce OLEA, a diagnostic, open-source, extensible Python library that provides easy-to-use tools for error analysis in the context of detecting offensive language in English.OLEA packages analyses and datasets proposed by prior scholarship, empowering researchers to build effective, explainable and generalizable offensive language classifiers.* The first three authors contributed equally.† Please direct inquiries about the library to this email.1 We use the term "offensive language" to encompass offensive language and hate speech.This paper contains censored offensive language examples.