Generic Mechanism for Reducing Repetitions in Encoder-Decoder Models
Ying Zhang, Hidetaka Kamigaito, Tatsuya Aoki, Hiroya Takamura, Manabu Okumura · 2021
Encoder-decoder models have been commonly used for many tasks such as machine translation and response generation.As previous research reported, these models suffer from generating redundant repetition.In this research, we propose a new mechanism for encoderdecoder models that estimates the semantic difference of a source sentence before and after being fed into the encoder-decoder model to capture the consistency between two sides.This mechanism helps reduce repeatedly generated tokens for a variety of tasks.Evaluation results on publicly available machine translation and response generation datasets demonstrate the effectiveness of our proposal.