Mask Guided Unsupervised Face Frontalization using 3D Morphable Model from Single-View Images
Jun Yin, Yuanquan Xu, Ningbo Wang, Yapeng Li, Siyu Guo · 2022
Face frontalization can be used for improving face recognition accuracy under large poses. Existing deep learning based methods have made great progress in face frontalization by using a large amount of front-profile image pairs for training. However, only a few public datasets provide such front-profile image pairs which are collected from constrained conditions. To address the problem of lacking training data, we propose a face frontalization framework combined with 3D Morphable Model (3DMM) that only adopts front images for training. Profile images with real facial textures are synthesized through 3DMM to form the front-profile image pairs from front face images. Furthermore, our proposed framework adopts masked background images to guide the generating position of front faces and maintain the background from inputs with dual discriminators. Experiment results demonstrate that our proposed framework can generate face images with photorealistic frontal view and maintain the consistency of face identity and retain the color tone as well as the background from input images using masked background images. Extensive experiments on Labeled Faces in the Wild (LFW) datasets show that our proposed framework achieves favorable performance on face recognition.