Tracking of global motion and facial expressions of a human face in image sequences
Marcel J. T. Reinders, F. A. Odijk, Jan C. A. van der Lubbe, Jan J. Gerbrands · Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE · 1993
We present a system in which the global motion (3D rotation and translation) and the local motion (facial expressions) of the face of a talking person are estimated automatically from an input image sequence. First, the shape of the facial features, such as eyes and mouth, are robustly extracted from the images. Then, based on the extracted shape of the facial features, the global motion and facial expressions are estimated. No human assistance is necessary throughout the process. The system relies on the use of a priori knowledge about the scene and facial motions. In the feature extraction scheme this a priori knowledge is modeled by a descriptive tree of the scene and geometric shape representations of the facial features. The global motion can be easily obtained from these facial features. In the local motion estimation scheme the a priori knowledge is modeled in a belief network, in which knowledge about muscle actuators is represented, e.g., the interactions between the muscle actuators and their visible manifestations. A few experiments are included to illustrate the system.