Fast and high-quality head reconstruction with explicit mesh and neural appearance
Haiyao Xiao, Chenglai Zhong, Yudong Guo · 2024
Digital humans are extensively utilized across various industries, including gaming, films, and AR/VR. The reconstruction of the human head from images or videos is a significant area of research. Existing methods could generally be categorized into two groups. One approach involves fitting a 3D Morphable Model (3DMM), which offers fast reconstruction results but often lacks precision and struggles with accurately capturing the hair region. The other approach employs neural implicit representations to model the human head, resulting in more detailed geometry but requiring a lengthy optimization process. Moreover, both methods have limitations in terms of practical application due to specific constraints on the data format used. To address these challenges, we propose a hybrid methodology. Our approach utilizes a triangular mesh to represent head geometry and optimizes per-vertex offsets while employing a well-designed neural appearance field to capture the texture. Leveraging this representation, we develop a differentiable renderer and perform joint optimization of geometry and texture. Experimental results demonstrate that our method achieves high-precision head reconstruction within 5 minutes based on head videos captured under arbitrary lighting conditions. The reconstruction encompasses the hair region, and all the required data can be captured exclusively using the front-facing camera of a smartphone.