A Framework for Audiovisual Dialog-based Human Computer Interaction
Georgios Galatas, Fillia S. Makedon · Journal of Multimedia Theory and Applications · 2014
In this paper, we describe a framework for audiovisual human computer interaction using a PC.The architecture of our system is based on recognized speech from both audio and visual information, and a dialog system that is capable of deriving the user's intentions, taking into account contextual information.The design of our system goes beyond the traditional paradigm of using vocal commands to control a computer which is usually application dependant.In order to do so it employs an adaptive module, able to select an appropriate grammar that suits the program used at a particular time.Furthermore, our system utilizes the visual modality in addition to audio, for increased word accuracy.We carried out a number of experiments during which we acquired promising results, therefore showing the great prospect of our framework.