Annotating Continuous Understanding in a Multimodal Dialogue Corpus
Gregory Aist, James F. Allen, William de Beaumont, Sergio R. Coria, Whitney M. Gegg-Harrison, Mary D. Swift · 2007
We describe an annotation scheme aimed at capturing continuous understanding behavior in a multimodal dialogue corpus involving referential description tasks. By using multilayer annotation at the word level as opposed to sentence level, we can better understand the role of continuous understanding in dialogue. To this end, we annotate referring expressions, spatial relations, and speech acts at the earliest word that clarifies the speaker’s intentions. Word-level annotation allows us to trace how referential expressions and actions are understood incrementally. Our corpus has intertwined language and actions which help identify the relationships between language usage, intention recognition, and contextual changes which in turn can be used to develop conversational agents that understand language in a continuous manner.