Impact of machine translation evaluation metrics on low resource languages
Sonithoi Ningombam, N. Donald Jefferson Thabah, Arindam Roy, Bipul Syam Purkayastha · 2025
Machine translation (MT) is one of the most prominent fields in computational linguistics and artificial intelligence (AI). Evaluating MT systems is essential, as it provides key insights into how translation systems and fine-tuned models have improved. While traditional human evaluations provide thorough insights, they are often resource-intensive and costly. To address these limitations, researchers have developed various automatic evaluation metrics that offer fast, cost-effective, and reliable assessments of translation quality. These metrics can be broadly classified into two categories: language-independent and language-dependent, with the latter emphasising the semantic context of sentences. This study examines prominent automatic evaluation metrics, including the language-independent BLEU and chrF, as well as the language-dependent BERT, RoBERTa, and S-BERT, on low-resource Indian Languages (namely: Meiteilon and Khasi). The discussion highlights their methodologies, strengths, weaknesses, and correlations with human evaluations.