Findings from shared tasks on hate speech detection: Performance patterns for low-resource languages
Koyel Ghosh, Saptarshi Saha, Thomas Mandl, Sandip Modha · Pattern Recognition Letters · 2025
In the digital era, social media has emerged as a powerful channel for expressing opinions, but online platforms have also become a breeding ground for hate speech targeting individuals based on color, caste, gender, sexual orientation, and political ideologies. Despite growing interest in automatic hate speech detection, existing research remains predominantly focused on English, underscoring a critical need to extend efforts to under-resourced languages. To bridge this gap, the HASOC (Hate Speech and Offensive Content Identification) shared task has been promoting multilingual hate speech research. In this paper, we present a brief overview of these four shared tasks (Assamese, Bengali, Bodo and English), datasets, participating systems, and their performance across standard evaluation metrics—precision, recall, accuracy, and macro F1 score. In addition, we analyze the inter-system agreement using Cohen’s κ and Fleiss’ κ , and investigate item-level difficulty through hardness analyses. Our findings offer valuable insights into the challenges and progress in multilingual hate speech detection, particularly for low-resource languages. This paper also serves as a model for the analysis of other results of large-scale experimentation with text classification systems. • Hate Speech Detection for social media in low-resource Languages and English is analyzed based on experiments for 4 languages. • Approaches for Hate Speech Detection and their performance are compared. • Item hardness of social media posts is analyzed and patterns for low-resource languages are compared. • Inter-system agreement or similarity is analyzed using Cohen’s κ and Fleiss’ κ .