A Roberta-Seq2Seq based model for Chinese text abstractive summarization
Junjie Sun, Xia Hou · 2022
In order to overcome the limitations of existing generative text summary generation algorithms in Chinese text and improve feature extraction ability of traditional deep learning models, a generative Chinese text summary generation model based on RoBERTa-Seq2Seq is proposed. The pre-training model RoBERTa is used to learn the dynamic meaning of current words in a specific context, so as to improve the semantic representation of words. Based on the Seq2Seq model, Luong Attention is used to further enhance global information. The experimental results show that our model’s ROUGE score is higher than some other traditional Seq2Seq models, which indicates that our RoBERTa-Seq2Seq based model can effectively improve the semantic representation ability of the generated summary in Chinese text and improve feature extraction ability of the traditional deep learning model.