Connectionist models in multimodal human-computer interaction
Alex H. Waibel, Paul Duchnowski · 1996
We present an overview of our laboratories' research on Multimodal Human-Computer Interfaces. By exploiting all available channels of human communication we aim to increase flexibility, robustness, and naturalness of human -computer interaction. The information sources we process include Speech-, Character-, and Gesture Recognition, Face- and Eye Tracking, Lipreading, and Sound Source Localization. Connectionist and hybrid techniques are used throughout. Introduction Recent developments in the computer and communication industries are rapidly increasing the amount and variety of information available to a wide and diverse audience. The multi-media nature of this data explosion, heralded by the concept of the "Information Superhighway", offers images, sound, text, etc. as the output presented to the information consumer. This is in stark contrast to the impoverished set of input options which are still largely limited to the keyboard and mouse. Attempts at the use of alternate modalit...