Translate and Classify: Improving Sequence Level Classification for English-Hindi Code-Mixed Data
Devansh Gautam, Kshitij Gupta, Manish Shrivastava · 2021
Code-mixing is a common phenomenon in multilingual societies around the world and is especially common in social media texts.Traditional NLP systems, usually trained on monolingual corpora, do not perform well on code-mixed texts.Training specialized models for code-switched texts is difficult due to the lack of large-scale datasets.Translating code-mixed data into standard languages like English could improve performance on various code-mixed tasks since we can use transfer learning from state-of-the-art English models for processing the translated data.This paper focuses on two sequence-level classification tasks for English-Hindi code mixed texts, which are part of the GLUECoS benchmark -Natural Language Inference and Sentiment Analysis.We propose using various pre-trained models that have been fine-tuned for similar English-only tasks and have shown state-of-the-art performance.We further finetune these models on the translated codemixed datasets and achieve state-of-the-art performance in both tasks.To translate English-Hindi code-mixed data to English, we use mBART, a pre-trained multilingual sequenceto-sequence model that has shown competitive performance on various low-resource machine translation pairs and has also shown performance gains in languages that were not in its pre-training corpus.