Understanding Multimodal Interaction by Exploiting Unification and Integration Rules
Håkan Johansson · DSpace repository (University of Tartu) · 2001
This paper presents a model for synergistic integration of multimodal speech and pen information.The model consists of an algorithm for matching and integrating interpretations of inputs from different modalities, as well as of a grammar that constrains integration.Integration proper is achieved by unifying feature structures.The integrator is part of a general framework for multimodal information systems with dialogue capabilities.Those parts of this framework that are relevant and affects the design of the integrator are also presented.