Abstractive Document Summarization with Summary-length Prediction
Jingun Kwon, Hidetaka Kamigaito, Manabu Okumura · 2023
Recently, we can obtain a practical abstractive document summarization model by finetuning a pre-trained language model (PLM).Since the pre-training for PLMs does not consider summarization-specific information such as the target summary length, there is a gap between the pre-training and fine-tuning for PLMs in summarization tasks.To fill the gap, we propose a method for enabling the model to understand the summarization-specific information by predicting the summary length in the encoder and generating a summary of the predicted length in the decoder in fine-tuning.Experimental results on the WikiHow, NYT, and CNN/DM datasets showed that our methods improve ROUGE scores from BART by generating summaries of appropriate lengths.Further, we observed about 3.0, 1,5, and 3.1 point improvements for ROUGE-1, -2, and -L, respectively, from GSum on the WikiHow dataset.Human evaluation results also showed that our methods improve the informativeness and conciseness of summaries.