Knowledge-Aware Graph-Enhanced GPT-2 for Dialogue State Tracking
Weizhe Lin, Bo-Hsiang Tseng, Bill Byrne · Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing · 2021
Dialogue State Tracking is central to multidomain task-oriented dialogue systems, responsible for extracting information from user utterances.We present a novel hybrid architecture that augments GPT-2 with representations derived from Graph Attention Networks in such a way to allow causal, sequential prediction of slot values.The model architecture captures inter-slot relationships and dependencies across domains that otherwise can be lost in sequential prediction.We report improvements in state tracking performance in Mul-tiWOZ 2.0 against a strong GPT-2 baseline and investigate a simplified sparse training scenario in which DST models are trained only on session-level annotations but evaluated at the turn level.We further report detailed analyses to demonstrate the effectiveness of graph models in DST by showing that the proposed graph modules capture inter-slot dependencies and improve the predictions of values that are common to multiple domains.