VAE-PGN based Abstractive Model in Multi-stage Architecture for Text Summarization
Hyungtak Choi, Lohith Ravuru, Tomasz Dryjański, Sunghan Rye, Dong‐Hyun Lee, Hojung Lee, In-Chul Hwang · 2019
This paper describes our submission to the TL;DR challenge.Neural abstractive summarization models have been successful in generating fluent and consistent summaries with advancements like the copy (Pointer-generator) and coverage mechanisms.However, these models suffer from their extractive nature as they learn to copy words from the source text.In this paper, we propose a novel abstractive model based on Variational Autoencoder (VAE) to address this issue.We also propose a Unified Summarization Framework for the generation of summaries.Our model eliminates non-critical information at a sentencelevel with an extractive summarization module and generates the summary word by word using an abstractive summarization module.To implement our framework, we combine submodules with state-of-the-art techniques including Pointer-Generator Network (PGN) and BERT while also using our new VAE-PGN abstractive model.We evaluate our model on the benchmark Reddit corpus as part of the TL;DR challenge and show that our model outperforms the baseline in ROUGE score while generating diverse summaries.