Deformable Feature Interaction Network and Graph Structure Reasoning for 3D Dense Alignment and Face Reconstruction
Jia Hao Deng, Xiaofei Li, Xing Wang, Xiangzheng Li · 2023
3D face reconstruction from large pose images is a long-standing and challenging problem. Existing methods based on 3DMM parameter regression with strong constraints only consider reducing the error between 68 landmarks and their ground truth, which may lead to insufficient learning about the landmark. In this paper, we propose a deformable feature interaction network (DFIN) for 3D dense alignment and face reconstruction. We design shallow and deep modules to extract semantic information at different levels, perform feature interaction, and improve the ability of the network to perform local detail feature extraction. We use deformable convolution properties to increase the convolution kernel's acceptance domain to learn the effective face area and the weights of the facial feature distribution, and the output features are enriched. Moreover, we propose a graph structure reasoning method by introducing a facial geometric consistency loss. Specifically, we selected the leftmost and rightmost cheeks, the chin, the tip of the nose, and the landmarks of the eyes. We triangulate the landmarks of the chin, eyes, and cheeks and calculate the geometric relationship between the center of gravity of the triangle and the tip of the nose to constrain the entire face. Extensive experiments on AFLW2000-3D and AFLW datasets demonstrate the validity of our method. Source codes are available at https://github.com/wricked520/DFIN.