Analysis, recognition and synthesis of facial gestures
Christoph von der Malsburg, Hai Hong · 2001
A paradigm of building a parameterized facial gesture model through statistical learning on training samples is presented in this dissertation. The 25 individual facial gestures we have studied are a set of common, voluntary, actable, and distinguishable facial expressions without any label of the emotional state. This is a facial expression model that addresses both geometry and texture representations. In the model, the geometry information is represented by motion vectors that are obtained by tracking feature nodes on the face through facial gesture image sequences with an automatic facial gesture tracking system we have developed based on Gabor wavelets. Before the tracking is taken, two alignments: person-specific alignment and person-independent general alignment have been performed on each sequence so that consistency of the motion vectors across different persons can be achieved. The texture information is represented by Gabor wavelet responses of the face images—jets. In our research the face region has been divided into a set of relatively independent component subregions and a grid structure of 133 feature nodes has been used to evenly and efficiently cover all the component subregions. Through analysis we have extracted a set of person-independent facial gesture components at the component subregions of the face from the training samples of individual facial gestures, which provides a basis for representing arbitrary facial gestures. Thereby our facial gesture model is parameterized by a number of parsimonious geometry parameters and texture parameters. In the model, a mapping function is further established to approximate the correlation relation between the geometry parameters and the texture parameters. Thus the model can accurately capture the variations with facial gestures in terms of both geometry information and texture information. This model can applied to cross-person facial gesture recognition and to the synthesis of a facial gesture sequence with only geometry information. The experimental results of recognition and synthesis have demonstrated the feasibility of our methods and the efficiency and accuracy of this model.