Situated Incremental Natural Language Understanding using a Multimodal, Linguistically-driven Update Model
Casey Kennington, Spyros Kousidis, David Schlangen · PUB – Publications at Bielefeld University (Bielefeld University) · 2014
A common site of language use is interactive dialogue between two people situated together in shared time and space.In this paper, we present a statistical model for understanding natural human language that works incrementally (i.e., does not wait until the end of an utterance to begin processing), and is grounded by linking semantic entities with objects in a shared space.We describe our model, show how a semantic meaning representation is grounded with properties of real-world objects, and further show that it can ground with embodied, interactive cues such as pointing gestures or eye gaze.