Summarization with Highway Condition Radom Pointer-Generator Network
Wenjun Zhao, Meina Song, E Haihong · 2018
Nowadays, Neural sequence-to-sequence models have become the common approach for abstractive text summarization. In this way, the abstract is closer to the natural language, not just the excerpt and recomposition of the original text. However, these models still have some shortcomings: They can't handle out of vocabulary(OOV) words well and they tend to repeat themselves or become confused. What's more, because of the selfcharacteristics of the recurrent neural network, it begins to degenerate when the depth is low (only two layers), which is not conducive to extracting deeper features to improve the accuracy of the model. In order to solve these problems, we have proposed a Highway Condition Radom Pointer-Generator Network(HCRPGN). We introduce the CRF layer to solve the duplication problem and use highway recurrent cell to optimize the neuron structure and prevent model degradation. We apply our model to the CNN/Daily Mail summarization task, outperforming the current abstractive state-of-the-art by at least 1 ROUGE points.