Research on Unknown Words Processing of Mongolian-Chinese Neural Machine Translation Based on Semantic Similarity
Hasigaowa, Siriguleng Wang · 2019 IEEE 4th International Conference on Computer and Communication Systems (ICCCS) · 2019
In this paper, the unknown words processing method in Mongolian-Chinese neural machine translation based on the similarity model and based on the Mongolian-Chinese alignment dictionary are studied. The unknown words processing method based on the similarity model uses the word vector to capture the characteristics of word semantics and grammatical information, calculates the semantic similarity between the unknown words and the words in the vocabulary, and selects the words with the closest semantics to replace all the unknown words. The unknown word processing method based on the Mongolian-Chinese alignment dictionary is to replace the unknown words with the word alignment information. Finally, the original corpus and the new corpus replacing the unknown words are merged and Training model. The final experiment showed that the translation's BLEU value was increased by 0.95 percentage points in the Mongolian-Chinese translation task.