A DAE-based Approach for Improving the Grammaticality of Summaries
Jie Huang, Yifan Jiang · 2021
While recent neural sequence-to-sequence models have achieved better and better Rouge performance in summarization task, there is little work emphasizing the grammaticality of the generated summaries. This paper proposes a simple method of pre-training the summarizer as a Denoising Autoencoder (DAE) to reduce the grammar errors. We apply two types of DAE: one is to simply reconstruct the input sentences where randomly sampled tokens are replaced with [MASK] elements, the other is to recover the words or phrases that are replaced with wrong expressions, namely designed for the grammatical error correction task. We evaluate the experiment outputs with three automatic metrics, and the result reveals better grammatical performance while having ROUGE scores higher than that of the baseline model.