Talking Head: Synthetic Video Facial Animation in MPEG-4.
Andriy Fedorov, T. Firsova, V. F. Kuriakin, Elena Martinova, K. V. Rodyushkin, V. G. Zhislina · 2003
We present a system for facial modeling and animation that aims at the generation of photo-realistic models and performance driven animation. It is practical implementation of MPEG-4 compliant Synthetic Video Facial Animation pipeline (Simple and Calibration Profiles with some modifications), which includes: facial features recognition & tracking on real video sequence; obtaining, encoding, network transfer and decoding of the facial animation parameters (FAPs); face model adaptation (calibration) and texturing by photos or video stream; synthetic 3D head animation and visualization. All pipeline stages, except model adaptation, operate in real time, fully automatically and produce appropriate results. To obtain the animation parameters new methods of robust online face tracking and more precise but off-line FAPs estimation are developed. Model-independent animation method was used. A number of new features including mouth contours special processing, eyelashes and “expression wrinkles” were added beyond standard to improve the resemblance of the Talking Head to the prototype person. The following libraries and applications were created to encapsulate “Talking Heads” original technologies: Face Tracking&Analysis Library, Images-to-Head Calibration application, Intel Facial Animation&Visualization Library. They are absolutely independent and could be used separately for various applications.