My Boli: Code-mixed Marathi-English Corpora, Pretrained Language Models and Evaluation Benchmarks
Tanmay Chavan, Omkar Gokhale, Aditya Kane, Shantanu Patankar, Raviraj Bhuminand Joshi · 2023
The research on code-mixed data is limited due to the unavailability of dedicated code-mixed datasets and pre-trained language models.In this work, we focus on the low-resource Indian language Marathi which lacks any prior work in code-mixing.We present L3Cube-MeCorpus, a large code-mixed Marathi-English (Mr-En) corpus with 10 million social media sentences for pretraining.We also release L3Cube-MeBERT and MeRoBERTa, code-mixed BERT-based transformer models pre-trained on MeCorpus.Furthermore, for benchmarking, we present three supervised datasets MeHate, MeSent, and MeLID for downstream tasks like codemixed Mr-En hate speech detection, sentiment analysis, and language identification respectively.These evaluation datasets individually consist of manually annotated ~12,000 Marathi-English code-mixed tweets.Ablations show that the models trained on this novel corpus significantly outperform the existing state-of-the-art BERT models.This is the first work that presents artifacts for code-mixed Marathi research.