ChatGPT is not a good indigenous translator
David Stap, Ali Araabi · 2023
This report investigates the continuous challenges of Machine Translation (MT) systems on indigenous and extremely low-resource language pairs.Despite the notable achievements of Large Language Models (LLMs) that excel in various tasks, their applicability to lowresource languages remains questionable.In this study, we leveraged the AmericasNLP competition to evaluate the translation performance of different systems for Spanish to 11 indigenous languages from South America.Our team, LTLAmsterdam, submitted a total of four systems including GPT-4, a bilingual model, finetuned M2M100, and a combination of finetuned M2M100 with kNN-MT.We found that even large language models like GPT-4 are not well-suited for extremely low-resource languages.Our results suggest that fine-tuning M2M100 models can offer significantly better performance for extremely low-resource translation.