Intelligent animated agents for interactive language training
Ron Cole, Tim Carmell, Pamela Connors, Mike Macon, Johan Wouters, Jacques de Villiers, Alice Tarachow, Dominic W. Massaro, Michael Cohen, Jonas Beskow, Jie Yang, Uwe Meier, Alex Waibel, Pat Stone, Alice Davis, Chris Soland, George E. Fortier · ACM SIGCAPH Computers and the Physically Handicapped · 1998
This report describes a three-year project, now eight months old, to develop interactive learning tools for language training with profoundly deaf children. The tools combine four key technologies: speech recognition, developed at the Oregon Graduate Institute; speech synthesis, developed at the University of Edinburgh and modified at OGI; facial animation, developed at University of California, Santa Cruz; and face tracking and speech reading, developed at Carnegie Mellon University. These technologies are being combined to create an intelligent conversational agent; a three-dimensional face that produces and understands auditory and visual speech. The agent has been incorporated into CSLU Toolkit, a software environment for developing and researching spoken language systems. We describe our experiences in bringing interactive learning tools to classrooms at the Tucker-Maxon Oral School in Portland, Oregon, and the technological advances that are required for this project to succeed.