Gaze enhanced speech recognition for truly hands-free and efficient text input during HCI
Matheus Vieira Portela, David Rozado · 2014
The performance of current speech recognition algorithms is well below that of human speech recognition, with high number of misrecognized words in quiet environments and degrading even further in noisy ones. Therefore, hands-free interaction remains a deeply frustrating experience. In this work, we present an innovative form of correcting misrecognized words during a speech recognition task by using gaze tracking technology in a multimodal approach. We propose to employ the user's gaze to point at misrecognized words and select appropriate alternatives. We compare the performance of this multimodal approach with traditional modalities of correcting words: usage of mouse and keyboard and usage of voice alone. The results of the user study show that whereas the proposed system is not as fast as using mouse and keyboard for correction, gaze enhanced correction significantly outperforms voice alone correction and is preferred by the users, offering a truly hands-free means of interaction.