A Text Summarization Generation Algorithm Based on the Improved GPT-2 Model
Xiangyang She, Xun Zhao · 2024
In response to the issues of polysemy in word vectors and inadequate contextual comprehension in traditional text summarization algorithms, this paper proposes a text summarization generation algorithm based on an improved GPT-2 model. The algorithm initially leverages a pre-trained language model, namely Bert, to acquire word vectors for enhanced semantic information. Subsequently, a time offset (TO) module is introduced before the Mask Multi-Head Self-Attention Mechanism to capture timing information in the text, thereby achieving a more accurate contextual understanding. Finally, decoding is performed using a greedy algorithm (GA), contributing to an improvement in the quality of the summaries. Experimental results demonstrate that on the NLPCC2017 and LCSTS datasets, the ROUGE metrics of this algorithm are consistently enhanced, effectively capturing key textual information and semantic correlations.