Quick Back-Translation for Unsupervised Machine Translation
Benjamin Brimacombe, Jiawei Zhou · 2023
The field of unsupervised machine translation has seen significant advancement from the marriage of the Transformer and the back-translation algorithm.The Transformer is a powerful generative model, and back-translation leverages Transformer's high-quality translations for iterative selfimprovement.However, the Transformer is encumbered by the run-time of autoregressive inference during back-translation, and backtranslation is limited by a lack of synthetic data efficiency.We propose a two-for-one improvement to Transformer back-translation: Quick Back-Translation (QBT).QBT re-purposes the encoder as a generative model, and uses encoder-generated sequences to train the decoder in conjunction with the original autoregressive back-translation step, improving data throughput and utilization.Experiments on various WMT benchmarks demonstrate that a relatively small number of refining steps of QBT improve current unsupervised machine translation models, and that QBT dramatically outperforms standard back-translation only method in terms of training efficiency for comparable translation qualities.