Modélisation statistique de la morphologie du visage pour la dermatologie et la chirurgie plastique
Florent Jousse · HAL (Le Centre pour la Communication Scientifique Directe) · 2022
Three-dimensional (3d) scans are essential in evaluating and following dermatological and cosmetic treatments. Unfortunately, the scanned body parts may be altered by artifacts from the reconstruction or unsought motions from the patients. For instance, facial expressions makes the 3d face scans improper for measurements. Hence, it is essential to develop methods to standardize the 3d scans to improve the measurement accuracy. Besides, evaluating treatments and labeling 3d scans is time-consuming, prone to errors, and undergoes rater's variability. For example, studies requiring the evaluation of skin aging or tissue sagging are easily distorted by inconsistencies between evaluators. Thus, it would be a significant leap for physicians to have data-driven algorithms that automatically evaluate the treatments. In this thesis, we develop statistical modeling methods for the face to address these problems.The first step is to register the 3d scans, which have different mesh connectivity and a different number of vertices. The Gaussian Process Morphable Models (GPMMs) framework can register 3d face scans with neutral expressions but struggle when facial expressions change. Hence, we propose a new kernel for GPMMs based on geodesic distances that allow more flexible and realistic deformations. Our new kernel allows for fitting the template mesh toward faces with different facial expressions. Also, our registration formulation uses weighted least squares to select areas such as hair that will not be registered. Furthermore, the recent advances in estimating geodesic distances make the extra computation time cost negligible in most applications.Then, we built a statistical shape model to quantify the skin sagging on the face. We use Partial Least Square Regression (PLSR) to find a linear relationship between the skin sagging score and geometric features on the face surface. The interpretability potential of linear models such as PLSR makes these methods ideal candidates for shape analysis on small data sets where generalization is essential. Furthermore, we propose visualization techniques to interpret the parameters of the PLSR model. Our jawline sagging model and the rater agree on 73% of the time, which is in the same range as the raters' variability. The visualization of the PLSR latent variables shows that the model has captured jawline sagging deformations that are coherent with the jawline sagging illustrated on the scales used by the physicians.Finally, we propose to use a morphable face model to neutralize facial expressions from 3d face scans. Unfortunately, in existing models, there is an entanglement between the deformations related to identity changes and those from facial expressions. Consequently, modifying a facial expression also modifies the person's identity, limiting such models' usefulness. We added an orthogonality penalization into the training procedure to address this issue, leading to a quasi-orthogonality between the expression and identity sub-spaces. The quasi orthogonality allows for a better disentanglement of facial expression deformations from face morphology. Averaging on all expressions, the neutralization with quasi-orthogonality produces face meshes that are 20% closer to the ground truth meshes. The effect of quasi-orthogonality is more visible on large amplitude facial expressions, such as opening the mouth. Our visualizations show very convincing facial expression neutralizations.