Zero-Shot Aspect-Based Scientific Document Summarization using Self-Supervised Pre-training
Amir Hasan Soleimani, Vassilina Nikoulina, Benoît Favre, Salah Ait Mokhtar · 2022
We study the zero-shot setting for the aspectbased scientific document summarization task.Summarizing scientific documents with respect to an aspect can remarkably improve document assistance systems and readers experience.However, existing large-scale datasets contain a limited variety of aspects, causing summarization models to over-fit to a small set of aspects and a specific domain.We establish baseline results in zero-shot performance (over unseen aspects and the presence of domain shift), paraphrasing, leave-one-out, and limited supervised samples experimental setups.We propose a self-supervised pre-training approach to enhance the zero-shot performance.We leverage the PubMed structured abstracts to create a biomedical aspect-based summarization dataset.Experimental results on the PubMed and FacetSum aspect-based datasets show promising performance when the model is pre-trained using unlabelled in-domain data. 1 * Work done while interning at NAVER LABS Europe. 1 github.com/asoleimanib/ZeroShotAspectBased