A study on the corpus expansion method of neural machine translation based on reverse transcription grammar
Zhen Sen Wu, Wang Guo · 2023
The current conventional corpus expansion methods mainly obtain the core words by mining the similarity of utterances to achieve the expansion of the corpus, which is better than the lack of decoding and encoding processing of the corpus data, resulting in the poor expansion effect. In this regard, a neural translation corpus expansion method based on reverse transcription grammar is proposed. The CYK decoding algorithm is combined with the move-in-reduction decoding algorithm to encode and decode the corpus data, and the difference and product of the corpus elements are quantified to realize the matching of utterance information, and finally the three-level filtering mechanism is used to obtain the core words under different classes to realize the corpus expansion. In the experiment, the proposed method is validated for the expansion effect. The analysis of the experimental results shows that the proposed method expands the corpus with high bilingual evaluation substitution values and has a more desirable expansion effect.