Tencent AI Lab Machine Translation Systems for WMT20 Chat Translation Task
Longyue Wang, Zhaopeng Tu, Xing Wang, Li Ding, Liang Ding, Shuming Shi · 2020
This paper describes the Tencent AI Lab's submission of the WMT 2020 shared task on chat translation in English⇔German.Our neural machine translation (NMT) systems are built on sentence-level, document-level, nonautoregressive (NAT) and pretrained models.We integrate a number of advanced techniques into our systems, including data selection, back/forward translation, larger batch learning, model ensemble, finetuning as well as system combination.Specifically, we proposed a hybrid data selection method to select highquality and in-domain sentences from out-ofdomain data.To better capture the source contexts, we exploit to augment NAT models with evolved cross-attention.Furthermore, we explore to transfer general knowledge from four different pre-training language models to the downstream translation task.In general, we present extensive experimental results for this new translation task.Among all the participants, our German⇒English primary system is ranked the second in terms of BLEU scores.