Redundancy Localization for the Conversationalization of Unstructured Responses
Sebastian Krause, Mikhail Kozhevnikov, Eric Malmi, Daniele Pighin · 2017
Conversational agents offer users a naturallanguage interface to accomplish tasks, entertain themselves, or access information.Informational dialogue is particularly challenging in that the agent has to hold a conversation on an open topic, and to achieve a reasonable coverage it generally needs to digest and present unstructured information from textual sources.Making responses based on such sources sound natural and fit appropriately into the conversation context is a topic of ongoing research, one of the key issues of which is preventing the agent's responses from sounding repetitive.Targeting this issue, we propose a new task, known as redundancy localization, which aims to pinpoint semantic overlap between text passages.To help address it systematically, we formalize the task, prepare a public dataset with fine-grained redundancy labels, and propose a model utilizing a weak training signal defined over the results of a passage-retrieval system on web texts.The proposed model demonstrates superior performance compared to a state-of-the-art entailment model and yields encouraging results when applied to a real-world dialogue. * Work performed during an internship at Google.User: What is Malaria?Agent: A disease caused by a plasmodium parasite, transmitted by the bite of infected mosquitoes.User: Is it a virus?Agent: Malaria is a parasitic infection spread by Anopheles mosquitoes.The Plasmodium parasite that causes Malaria is neither a virus nor a bacterium -it is a single-celled parasite that multiplies in red blood cells of humans as well as in the mosquito intestine.