Bi-Directional Neural Machine Translation with Synthetic Parallel Data
Xing Hua Niu, Michael Denkowski, Marine Jacinthe Carpuat · 2018
Despite impressive progress in highresource settings, Neural Machine Translation (NMT) still struggles in lowresource and out-of-domain scenarios, often failing to match the quality of phrasebased translation.We propose a novel technique that combines back-translation and multilingual NMT to improve performance in these difficult cases.Our technique trains a single model for both directions of a language pair, allowing us to back-translate source or target monolingual data without requiring an auxiliary model.We then continue training on the augmented parallel data, enabling a cycle of improvement for a single model that can incorporate any source, target, or parallel data to improve both translation directions.As a byproduct, these models can reduce training and deployment costs significantly compared to uni-directional models.Extensive experiments show that our technique outperforms standard backtranslation in low-resource scenarios, improves quality on cross-domain tasks, and effectively reduces costs across the board.