Low-Resource Dialogue Summarization with Domain-Agnostic Multi-Source Pretraining
Yicheng Zou, Bolin Zhu, Xingwu Hu, Tao Gui, Qi Zhang · Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing · 2021
With the rapid increase in the volume of dialogue data from daily life, there is a growing demand for dialogue summarization.Unfortunately, training a large summarization model is generally infeasible due to the inadequacy of dialogue data with annotated summaries.Most existing works for low-resource dialogue summarization directly pretrain models in other domains, e.g., the news domain, but they generally neglect the huge difference between dialogues and conventional articles.To bridge the gap between out-of-domain pretraining and indomain fine-tuning, in this work, we propose a multi-source pretraining paradigm to better leverage the external summary data.Specifically, we exploit large-scale in-domain nonsummary data to separately pretrain the dialogue encoder and the summary decoder.The combined encoder-decoder model is then pretrained on the out-of-domain summary data using adversarial critics, aiming to facilitate domain-agnostic summarization.The experimental results on two public datasets show that with only limited training data, our approach achieves competitive performance and generalizes well in different dialogue scenarios.