Incorporating Distributions of Discourse Structure for Long Document Abstractive Summarization
Dongqi Pu, Yifan Wang, Vera Demberg · 2023
For text summarization, the role of discourse structure is pivotal in discerning the core content of a text.Regrettably, prior studies on incorporating Rhetorical Structure Theory (RST) into transformer-based summarization models only consider the nuclearity annotation, thereby overlooking the variety of discourse relation types.This paper introduces the 'RSTformer', a novel summarization model that comprehensively incorporates both the types and uncertainty of rhetorical relations.Our RST-attention mechanism, rooted in document-level rhetorical structure, is an extension of the recently devised Longformer framework.Through rigorous evaluation, the model proposed herein exhibits significant superiority over state-of-theart models, as evidenced by its notable performance on several automatic metrics and human evaluation. 1