History-Aware Hierarchical Transformer for Multi-session Open-domain Dialogue System
Tong Zhang, Yong Liu, Boyang Li, Zhiwei Zeng, Pengwei Wang, Yuan You, Chunyan Miao, Lizhen Cui · 2022
With the evolution of pre-trained language models, current open-domain dialogue systems have achieved great progress in conducting one-session conversations.In contrast, Multi-Session Conversation (MSC), which consists of multiple sessions over a long term with the same user, is under-investigated.In this paper, we propose History-Aware Hierarchical Transformer (HAHT) for multi-session opendomain dialogue.HAHT maintains a long-term memory of history conversations and utilizes history information to understand current conversation context and generate well-informed and context-relevant responses.Specifically, HAHT first encodes history conversation sessions hierarchically into a history memory.Then, HAHT leverages historical information to facilitate the understanding of the current conversation context by encoding the history memory together with the current context with attention-based mechanisms.Finally, to explicitly utilize historical information, HAHT uses a history-aware response generator that switches between a generic vocabulary and a historyaware vocabulary.Experimental results on a large-scale MSC dataset suggest that the proposed HAHT model consistently outperforms baseline models.Human evaluation results support that HAHT generates more human-like, context-relevant and history-relevant responses than baseline models.