Topic Augmented Neural Network for Short Text Conversation.
Yu Wu, Wei Wu, Zhoujun Li, Ming Zhou · arXiv (Cornell University) · 2016
Message response matching is an important task within retrieval-based chatbots. We present a topic augmented neural network(TANN), comprising a sentence embedding layer, a topic embedding layer, and a matching layer, to match messages and response candidates. TANN inherits the benefits of neutral networks on matching sentence pairs, and leverages extra topic information and their corresponding weights as prior knowledge into a matching process. In TANN, the sentence embedding layer embeds an input message and a response into a vector space, while the topic embedding layer forms a topic vector by a linear combination of the embedding of topic words whose weights are determined by both themselves and the message vector. The message vector, the response vector, and the topic vector are then fed to the matching layer to calculate a matching score. The extensive evaluation of TANN, using large human annotated data sets, shows that TANN outperforms simple neutral network methods, while beating other typical matching models with a large margin.