Automated language acquisition in multimodal environment
D. Nagy, A. Medl, James L. Flanagan · 2002
The article describes a real-time laboratory prototype system that adaptively acquires multimodal language and uses it for natural style human-machine communication. Methods for building information theoretic networks and dialogue control are presented. Example interactions are presented and results are analyzed for two modalities: speech and mouse (pen) pointer. Future work is outlined, with special emphasis on the application of multimodal language learning in information transformation.