User Affect and No-Match Dialogue Scenarios
Joseph B. Wiggins, Mayank Kulkarni, Wookhee Min, Kristy Elizabeth Boyer, Bradford Mott, Eric Wiebe, James C. Lester · 2018
Recent years have seen significant advances in natural language dialogue management and a growing recognition that multimodality can inform dialogue policies. A key dialogue policy problem is presented by 'no-match' scenarios, in which the dialogue system receives a user utterance for which no matching response is found. This paper reports on a study of the 'no-match' problem in the context of a dialogue agent embedded within a game-based learning environment. We investigate how users' facial expressions exhibited in response to the agent's no-match utterances predict the users' opinion of the agent after the interaction has completed. The results indicate that models incorporating users' facial expressions following no-match utterances are highly predictive of user opinion and significantly outperform baseline models. This work represents a key step toward affect-informed dialogue systems whose policies are informed by users' affective expression.