Advancing Hate Speech Detection in Indonesian Code-Mixed Using Large Language Models (LLMS): Investigating Models and Strategies
R.G. Alam Nusantara Putra Herlambang, Endang Wahyu Pamungkas · 2025
Hate speech, explained as communication targeting people or comunities based on specific features like ethnicity, color, gender, or even religion, come to be more widespread due to the quick development of social media. In Indonesia, hate speech detection is particularly challenging owing to linguistic variety and frequent code-mixing on social media platforms. This research focuses on testing the efficacy of various Large Language Model (LLM) and non-LLM models in identifying hate speech on two variations of codemixed hate speech Javanese-Indonesia and SundaneseIndonesian obtained from Indonesia social media. We evaluated thirteen models -including Large Language Models (LLMs: Gemma-2B, Llama-3B-I, DeepSeek-R1 variants) and non-LLMs (CodeMix-RoBERTa, IndoBERT)-on manually annotated (Data V1, V2) and synthetically generated datasets (Data Convert). Two experimental benchmarks were conducted: first training on V2, testing on V1 and data convert; second training on convert data, testing on V1 and V2. Gemma-2B achieved state-of-the-art performance on JavaneseIndonesian data (F1: 0.94, Accuracy: 0.93), surpassing DeepSeek-R1-Llama and Llama-3B-I. Non-LLM architectures, notably CodeMix-RoBERTa (F1: 0.82) and XLM-R-L (F1: 0.81), demonstrated competitive efficacy under synthetic training, while smaller models exhibited significant performance degradation on Sundanese-Indonesian data, reflecting resource-dependent limitations. Synthetic data augmentation yielded a 3.9 % average$F 1$improvement across models. These findings underscore the dual role of synthetic data in mitigating low-resource language challenges and the dominance of LLMs in code-mixed contexts. Future research should prioritize dataset expansion, optimized synthetic data generation, and hybrid architectures to navigate linguistic complexity in multilingual online discourse.