Non-Parametric Unsupervised Domain Adaptation for Neural Machine Translation
Xin Wei Zheng, Zhirui Zhang, Shujian Huang, Boxing Chen, Jun Xie, Weihua Luo, Jiajun Chen · 2021
Recently, kNN-MT (Khandelwal et al., 2020) has shown the promising capability of directly incorporating the pre-trained neural machine translation (NMT) model with domainspecific token-level k-nearest-neighbor (kNN) retrieval to achieve domain adaptation without retraining.Despite being conceptually attractive, it heavily relies on high-quality indomain parallel corpora, limiting its capability on unsupervised domain adaptation, where indomain parallel corpora are scarce or nonexistent.In this paper, we propose a novel framework that directly uses in-domain monolingual sentences in the target language to construct an effective datastore for k-nearest-neighbor retrieval.To this end, we first introduce an autoencoder task based on the target language, and then insert lightweight adapters into the original NMT model to map the token-level representation of this task to the ideal representation of translation task.Experiments on multi-domain datasets demonstrate that our proposed approach significantly improves the translation accuracy with target-side monolingual data, while achieving comparable performance with back-translation.