Dual knowledge distillation for bidirectional neural machine translation
Huaao Zhang, Shigui Qiu, Shilong Wu · 2021
Building strong and robust neural machine translation systems needs large amount of high-quality parallel corpora. However, most of language pairs are limited in quantity, coverage and quality. In order to make full use of parallel sentences, we proposed dual knowledge distillation and bidirectional neural machine translation, leveraging one side information to boost the performance of another side. Our approach which only takes advantage of parallel corpora is supplementary to semi-supervised technology e.g. back-translation. Experiments on several low resource dataset show our approach achieves significant improvement over strong baseline. Our method combined with back-translation achieves state-of-the-art scores of 31.88 and 39.01 on IWSLT14 English-German and German-English translation tasks. Experiments on several low resource dataset show our approach achieves significant improvement over strong baseline. Our method combined with back-translation achieves state-of-the-art scores of 31.88 and 39.01 on IWSLT14 English-German and German-English translation tasks.