WinkTalk: a multimodal speech synthesis interface linking facial expressions to expressive synthetic voices
Éva Székely, Zeeshan Ahmed, João P. Cabral, Julie Carson-Berndsen · KTH Publication Database DiVA (KTH Royal Institute of Technology) · 2012
This paper describes a demonstration of the WinkTalk system, which is a speech synthe- sis platform using expressive synthetic voices. With the help of a webcamera and facial ex- pression analysis, the system allows the user to control the expressive features of the syn- thetic speech for a particular utterance with their facial expressions. Based on a person- alised mapping between three expressive syn- thetic voices and the users facial expressions, the system selects a voice that matches their face at the moment of sending a message. The WinkTalk system is an early research pro- totype that aims to demonstrate that facial expressions can be used as a more intuitive control over expressive speech synthesis than manual selection of voice types, thereby con- tributing to an improved communication expe- rience for users of speech generating devices.