Generalizing Back-Translation in Neural Machine Translation
Miguel Graça, Yunsu Kim, Julian Schamper, Shahram Khadivi, Hermann Ney · 2019
Back-translation -data augmentation by translating target monolingual data -is a crucial component in modern neural machine translation (NMT).In this work, we reformulate back-translation in the scope of crossentropy optimization of an NMT model, clarifying its underlying mathematical assumptions and approximations beyond its heuristic usage.Our formulation covers broader synthetic data generation schemes, including sampling from a target-to-source NMT model.With this formulation, we point out fundamental problems of the sampling-based approaches and propose to remedy them by (i) disabling label smoothing for the target-to-source model and (ii) sampling from a restricted search space.Our statements are investigated on the WMT 2018 German ↔ English news translation task.