Cache-based language model adaptation using visual attention for ASR in meeting scenarios
Neil J. Cooke, Martin J. Russell · 2009
In a typical group meeting involving discussion and col-laboration, people look at one another, at shared informa-tion resources such as presentation material, and also at nothing in particular. In this work we investigate whether the knowledge of what a person is looking at may improve the performance of Automatic Speech Recognition (ASR). A framework for cache Language Model (LM) adaptation is proposed with the cache based on a person’s Visual At-tention (VA) sequence. The framework attempts to measure the appropriateness of adaptation from VA sequence charac-teristics. Evaluation on the AMI Meeting corpus data shows reduced LM perplexity. This work demonstrates the poten-tial for cache-based LM adaptation using VA information in large vocabulary ASR deployed in meeting scenarios.