Research on the key technologies of corpus preprocessing in Mongolian-Chinese SMT
Jinting Li, Hongxu Hou, Jing Wu, Hongbin Wang, Wen-ting Fan · 2016
The traditional preprocessing method in morphology analysis uses Mongolian suffix segmentation and stemming.But there exists many cases in Mongolian.If the case is not processed, the corpus will suffer from data sparse problem and lead to poor translation performance.Therefore, we summarize and research the existing corpus preprocessing method, and focus on the effect of case processing, in order to improving the performance of Mongolian-Chinese SMT by analyzing Mongolian morphological.Experiments show improvements of about 3.22 relative in the BLEU score of SMT over baseline system 1 by optimizing the preprocessing method.