AgreeSum: Agreement-Oriented Multi-Document Summarization
Richard Yuanzhe Pang, Adam D. Lelkes, Vinh Tran, Cong Yu · 2021
We aim to renew interest in a particular multidocument summarization (MDS) task which we call AgreeSum: agreement-oriented multidocument summarization.Given a cluster of articles, the goal is to provide abstractive summaries that represent information common and faithful to all input articles.Given the lack of existing datasets, we create a dataset for AgreeSum, and provide annotations on article-summary entailment relations for a subset of the clusters in the dataset.We aim to create strong baselines for the task by applying the top-performing pretrained singledocument summarization model PEGASUS onto AgreeSum, leveraging both annotated clusters by supervised losses, and unannotated clusters by T5-based entailment-related and language-related losses.Compared to other baselines, both automatic evaluation and human evaluation show better article-summary and cluster-summary entailment in generated summaries.On a separate note, we hope that our article-summary entailment annotations contribute to the community's effort in improving abstractive summarization faithfulness.