A Generative Text Summarization Method Based on mT5 and Large Language Models
Feng Liu, Caiquan Xiong · 2023
A single pre-trained language model (such as mT5) still often fails to capture key information and cover sufficient content for the text summarization task. We propose a generative text summarization method (mT5-LLM) based on mT5 and a large language model. The method combines the best of both models to improve the accuracy and fluency of result summaries as follows: firstly, pre-training the mT5 model and fine-tuning it on the text summarization task so that it can generate summaries related to the input text; secondly, we use the fine-tuned mT5 model to generate a preliminary summary; thirdly, we use a large language model such as ChatGPT to modify and improve the preliminary summary according to the in-context learning. Besides, we employ some prompt engineering methods to produce a more accurate and fluent summary. After several rounds of interaction and modification, the last generated summary is taken as the final result. The proposed method is evaluated by experiments on the LCSTS public dataset. The results show that our proposed method achieved a higher ROUGE score than the traditional model algorithm. Ablation studies also show that our method effectively improves the quality of the abstract.