Leveraging Domain Knowledge at Inference Time for LLM Translation: Retrieval versus Generation
Bryan Li, Jiaming Luo, Eleftheria Briakou, Colin Cherry · 2025
While large language models (LLMs) have been increasingly adopted for machine translation (MT), their performance for specialist domains such as medicine and law remains an open challenge.Prior work has shown that LLMs can be domain-adapted at test-time by retrieving targeted few-shot demonstrations or terminologies for inclusion in the prompt.Meanwhile, for general-purpose LLM MT, recent studies have found some success in generating similarly useful domain knowledge from an LLM itself, prior to translation.Our work studies domain-adapted MT with LLMs through a careful prompting setup, finding that demonstrations consistently outperform terminology, and retrieval consistently outperforms generation.We find that generating demonstrations with weaker models can close the gap with larger model's zero-shot performance.Given the effectiveness of demonstrations, we perform detailed analyses to understand their value.We find that domainspecificity is particularly important, and that the popular multi-domain benchmark is testing adaptation to a particular writing style more so than to a specific domain.{"Risiken": ["risks"], "Poulvac FluFend H5N3 RG": ["Poulvac FluFend H5N3 RG"], [ "verbunden": ["connected", "associated"]} Welche Risiken sind mit Luminity verbunden?What risks are associated with Luminity?Wie wikrt Poulvac FluFend H5N3 RG? How does Poulvac FluFend H5N3 RG work?Welche Risiken sind mit Procoralan verbunden?What risks are associated with Procoralan?External DB de en Retrieve {"Risiken": ["risks"], "Poulvac FluFend H5N3 RG": ["Poulvac FluFend H5N3 RG"], [ "verbunden": ["connected"]}