Synergy Effects in Natural Language-Based Multimodal Interaction
Ivan Bretan, Jussi Karlgren · 1994
: We discuss the synergetic effects that can be obtained in an integrated multimodal interface framework comprising on one hand a visual language-based modality and on the other natural language analysis and generation components. Besides a visual language with high expressive power, the framework includes a cross-modal translation mechanism which enables mutual illumination of interface language syntax and semantics. Special attention has been payed to how to address problems with robustness and pragmatics through unconventional methods which aim to enable user control of the discourse management process. Keywords: Multimodal interaction, Natural language interfaces. Also Published: This paper also appeared in the Proceedings of the ERCIM 1993 Workshop on Multimodal Human-Computer Interaction, Nancy, France, November 1993. ii 1 Introduction The main point with providing interaction through multiple modalities in the first place is that the total usability thus obtained is grea...