Building a portable gesture-to-audio/visual speech system.
Sidney Fels, Robert Pritchard, Eric Vatikiotis‐Bateson · 2008
We have constructed an easy-to-use portable, wearable gesture-to-speech system based on the Glove-TalkII and GRASSP gesture-controlled speech systems and a vizeme based face-synthesizer. Our new portable system is called a Digital Ventriloquized Actor (DIVA) and refines the use of the formant speech synthesizer. Using a DIVA, a user can speak using hand gestures mapped to both synthetic sound and face using a mapping function that preserves gesture trajectories. By making DIVAs portable and self-contained, speakers can communicate with others in the community and perform in new music/theatre stage productions. DIVA performers also allow us to study the relationship between visible gestures and speech/song production. Index Terms: speech synthesis, gesture, face synthesis, wearable computing