Correlation-Driven 3D Facial Expression Blending with Stacked Emotion Prediction
Thi Chau, Anh Duc Dam, Dang-Ha Nguyen · 2024
3D facial reconstruction from monocular images is a challenging task, with applications ranging from virtual avatars to emotion analysis. While recent methods have shown impressive results in reconstructing facial geometry, accurately capturing and manipulating subtle emotional expressions remains a challenge. This paper presents a novel pipeline for generating 3D faces with new, blended emotional expressions by combining existing expressions. Our pipeline consists of two main phases: (1) Emotion Prediction, using a stacked model to classify emotions from a single image, and (2) Correlation-Driven Expression Blending, a novel method for combining emotional parameters based on their correlations with specific emotions. We demonstrate the effectiveness of our approach in generating realistic and emotionally accurate 3D facial expressions, outperforming traditional linear interpolation techniques. Our method is particularly effective in capturing subtle emotional nuances, especially in the context of Asian facial features, where expressions are often conveyed more subtly through eye movements. However, the accuracy of the emotion prediction phase is crucial for the success of the blending phase, as mislabeling the initial emotion can lead to inaccurate or unexpected results.