Summary-Source Proposition-level Alignment: Task, Datasets and Supervised Baseline
Ori Ernst, Ori Shapira, Ramakanth Pasunuru, Michael Lepioshkin, Jacob H. Goldberger, Mohit Bansal, Ido Dagan · 2021
Aligning sentences in a reference summary with their counterparts in source documents was shown as a useful auxiliary summarization task, notably for generating training data for salience detection.Despite its assessed utility, the alignment step was mostly approached with heuristic unsupervised methods, typically ROUGE-based, and was never independently optimized or evaluated.In this paper, we propose establishing summary-source alignment as an explicit task, while introducing two major novelties: (1) applying it at the more accurate proposition span level, and (2) approaching it as a supervised classification task.To that end, we created a novel training dataset for proposition-level alignment, derived automatically from available summarization evaluation data.In addition, we crowdsourced dev and test datasets, enabling model development and proper evaluation.Utilizing these data, we present a supervised proposition alignment baseline model, showing improved alignmentquality over the unsupervised approach.