Dense Optical Flow Variation Based 3D Face Reconstruction from Monocular Video
Shan Wang, Xukun Shen, Jiaqing Liu · 2018
This paper presents a method for reconstructing 3D face expressions from monocular video sequences. Unlike previous approaches we don't require any prior face models, nor a large collection of images with diverse variation of poses and illuminations. Instead, we leverage a monocular video sequence without any restrictions. We formulate the 3D face reconstruction as an energy minimization problem integrated with dense optical flow variation, as rigid as possible(ARAP) constraint, spatial and temporal constraints. This paper offers the first dense optical flow variational approach to the problem of 3D reconstruction of non-rigid face expressions from a monocular video. Dense optical flow variation cost substitutes for photo consistency cost to enhance the reconstruction of exaggerated expressions. A generic 3D face template mesh and a simple 3D warping algorithm allow us to reconstruct a true 3D face mesh, relax the constraints of diverse views or illuminations and also avoid the dependency of the quality of prior face models, such as the facial expressions or face races varieties limitation. Finally, we use a per-pixel shape-from-shading(SFS) algorithm to estimate the fine-scale geometry details such as wrinkles to further improve the reconstruction fidelity. Given unconstrained monocular RGB videos, our method reconstructs wrinkle-level 3D face model, without the need for any prior models or diverse capture conditions.