High-fidelity facial reconstruction from a single photo using photo-realistic rendering
Mariana Dias, Alexis Roche, Margarida Fernandes, Verónica Orvalho · 2022
We propose a fully automated method for realistic 3D face reconstruction from a single frontal photo that produces a high-resolution head mesh and a diffuse map. The photo is input to a convolutional neural network that estimates the weights of a morphable model to produce an initial head shape that is further adjusted through landmark-guided deformation. Two key features of the method are: 1) the network is exclusively trained on synthetic photos that are photo-realistic enough to learn real shape predictive features, making it unnecessary to train with real facial photos and corresponding 3D scans; 2) the landmarking statistical errors are incorporated in the reconstruction for optimal accuracy. While the method is based on a limited amount of real data, we show that it robustly and quickly performs plausible face reconstructions from real photos.