Machine Translation with Large Language Models: Prompting, Few-shot Learning, and Fine-tuning with QLoRA
Xuan Zhang, Navid Rajabi, Kevin Duh, Philipp Koehn · 2023
While large language models have made remarkable advancements in natural language generation, their potential in machine translation, especially when fine-tuned, remains underexplored.In our study, we conduct comprehensive experiments, evaluating 15 publicly available language models on machine translation tasks.We compare the performance across three methodologies: zero-shot prompting, fewshot learning, and fine-tuning.Central to our approach is the use of QLoRA, an efficient finetuning method.On French-English, QLoRA fine-tuning outperforms both few-shot learning and models trained from scratch.This superiority is highlighted in both sentence-level and document-level translations, with a significant BLEU score improvement of 28.93 over the prompting method.Impressively, with QLoRA, the enhanced performance is achieved by finetuning a mere 0.77% of the model's parameters.