A Study on Non-Autoregressive Mongolian-Chinese Neural Machine Translation for Multilingual Pre-Training
Xiaoli Zheng, Yonghong Tian, Chang Ma, Kangkang Sun · 2024
Aiming at the problem that most of the current Mongolian-Chinese Neural Machine Translation (NMT) adopts autoregressive generation, which is prone to error accumulation and slow generation speed, we propose the study of non-autoregressive Mongolian-Chinese NMT. However, non-autoregressive generation generally suffers from the phenomenon of poor translation quality, and its performance depends largely on the quantity and quality of data, so we resort to the multilingual pre-trained model CeMAT to assist the non-autoregressive Mongolian-Chinese NMT task, and then use CeMAT-based autoregressive Mongolian-Chinese NMT as teacher model, adopt multi-level knowledge distillation to train the CeMAT-based non-autoregressive Mongolian-Chinese NMT model, and finally incorporating Graph Convolutional Networks(GCN) to supplement the semantic information into the word embedding layer by modelling word-to-word relationships. The experimental results show that the methods used obtains a BLEU value of +9.99 and an increase in translation speed by a factor of 1.94 compared to the autoregressive model, i.e., it ensures an increase in the translation speed of the research model along with a significant increase in the quality of the translated text.