Zero-Resource Neural Machine Translation with Monolingual Pivot Data
Anna Currey, Kenneth Heafield · 2019
Zero-shot neural machine translation (NMT) is a framework that uses source-pivot and target-pivot parallel data to train a sourcetarget NMT system.An extension to zeroshot NMT is zero-resource NMT, which generates pseudo-parallel corpora using a zeroshot system and further trains the zero-shot system on that data.In this paper, we expand on zero-resource NMT by incorporating monolingual data in the pivot language into training; since the pivot language is usually the highest-resource language of the three, we expect monolingual pivot-language data to be most abundant.We propose methods for generating pseudo-parallel corpora using pivotlanguage monolingual data and for leveraging the pseudo-parallel corpora to improve the zero-shot NMT system.We evaluate these methods for a high-resource language pair (German-Russian) using English as the pivot.We show that our proposed methods yield consistent improvements over strong zero-shot and zero-resource baselines and even catch up to pivot-based models in BLEU (while not requiring the two-pass inference that pivot models require).