Seq2seq Deep Learning Method for Summary Generation by LSTM with Two-way Encoder and Beam Search Decoder
Gábor Szücs, Dorottya Huszti · 2019
In this paper a deep neural network architecture is proposed for abstractive summarization, which aims generating brief summaries from long text documents. Unlike default sequence-to-sequence model with one-way encoder, the proposed solution has a two-way encoder with state-of-the-art recurrent neural networks type, LSTM (Long Short-Term Memory). One of the LSTMs starts from the beginning of the series, and the other from the end, so the final state is created by combining the inner states of them. Furthermore, two types of decoder were used, one for training, and an extended one for inferencing. The extended decoder is based on the beam search idea, which retains more than one candidate when generating sequence elements. The final prediction of the decoder will be the sequence with the highest probability. In the system the attention mechanism and appropriate embedding layer also help with text generation. Experiments are presented on the Newsroom dataset with relational article and summary pairs.