Length Adaptive Regularization for Retrieval-based Chatbot Models
Disen Wang, Hui Min Fang · 2020
Chatbots aim to mimic real conversations between humans. They have started playing an increasingly important role in our daily life. Given past conversations, a retrieval-based chatbot model selects the most appropriate response from a pool of candidates. Intuitively, based on the nature of the conversations, some responses are expected to be long and informative while others need to be more concise. Unfortunately, none of the existing retrieval-based chatbot models have considered the effect of response length. Empirical observations suggested the existing models over-favor longer candidate responses, leading to sub-optimal performance.