Transformer and seq2seq model for Paraphrase Generation
Elozino Egonmwan, Yllias Chali · 2019
Paraphrase generation aims to improve the clarity of a sentence by using different wording that convey similar meaning.For better quality of generated paraphrases, we propose a framework that combines the effectiveness of two models -transformer and sequence-tosequence (seq2seq).We design a two-layer stack of encoders.The first layer is a transformer model containing 6 stacked identical layers with multi-head self-attention, while the second-layer is a seq2seq model with gated recurrent units (GRU-RNN).The transformer encoder layer learns to capture long-term dependencies, together with syntactic and semantic properties of the input sentence.This rich vector representation learned by the transformer serves as input to the GRU-RNN encoder responsible for producing the state vector for decoding.Experimental results on two datasets-QUORA and MSCOCO using our framework, produces a new benchmark for paraphrase generation.