Automated lip synchronized speech driven facial animation
Zeki Melek, Lale Akarun · 2002
Talking 3D synthetic faces are now used in many applications involving human-computer interaction. The lip synchronization of the faces are mostly done mechanically by computer animators. Although there is some work done on automated lip synchronized facial animation, these studies are mostly based on text input. We use speech in Turkish as an input to generate lip synchronized facial animation. Speakers' recorded voice is converted into lip shape classes on the 3D model. Voice is analyzed and classified using a training set. Lip animation is facilitated using facial muscles and the jaw. Facial muscles are modelled onto our facial model. For more realistic facial animation, facial tissue is modelled as well, and the interactions between epidermis, subcutaneous layer and bone are taken into account. Natural-looking facial animation is achieved in real-time on a personal computer. We also show that our system is compatible with MPEG4.