Multi-Document Summarization with Centroid-Based Pretraining
Ratish Puduppully, Parag Jain, Nancy F. Chen, Mark J. Steedman · 2023
In Multi-Document Summarization (MDS), the input can be modeled as a set of documents, and the output is its summary.In this paper, we focus on pretraining objectives for MDS.Specifically, we introduce a novel pretraining objective, which involves selecting the ROUGEbased centroid of each document cluster as a proxy for its summary.Our objective thus does not require human written summaries and can be utilized for pretraining on a dataset consisting solely of document sets.Through zero-shot, few-shot, and fully supervised experiments on multiple MDS datasets, we show that our model Centrum is better or comparable to a state-ofthe-art model.We make the pretrained and finetuned models freely available to the research community 1 .