Pose Invariant 3D Facial Landmark Detection Via Pose Normalization and Deep Regression
Jingchen Zhang, Kangkang Gao, Qijun Zhao, Daning Wang · 2020
Despite the rapid development of detecting facial landmarks on 2D images in the past decade, detecting the landmarks on 3D faces under varying poses is still challenging. Existing methods rely either on pose-invariant feature descriptors, which are vulnerable to missing data (e.g., due to self-occlusion at large poses), or on multi-view landmark models, which are complicated to apply. In this paper, instead, we propose to first normalize the 3D face to frontal pose and then use a single deep model to regress the facial landmarks. We adapt a pose estimation network to estimate the yaw and pitch angles of the input 3D face based on its depth image, and then rotate the face to frontal pose and encode it as a position map. We employ another deep network to predict the location of landmarks on the position map, which is projected back onto the input 3D face to get the final detection results. Evaluation experiments using three public datasets (FRGCv2, UND and Bosphorus) prove the superiority of the proposed method for pose-invariant 3D facial landmark detection.