3d face surface and texture synthesis from 2d landmarks of a single face sketch
Ulrich Neumann, Tanasai Sucontphunt · 2012
Synthesizing a 3D human face surface and texture from a drawing is a challenging problem. The problem involves inferring a 3D model and its color information from 2D information. In contrast, human beings have the natural ability to reconstruct 3D models from a drawing in their mind effortlessly. This skill is built up from years of experience in mapping the perceived 2D information to the 3D model in the actual scene. By imitating this mapping process, this work illustrates an approach to reconstruct a 3D human face from just the 2D facial landmarks gathering from a sketch image. The 2D facial landmarks of the sketch image contain enough information for this process because they semantically represent a facial structure that can be recognizable as a human face. The approach also exploits the perspective distortion of the sketch image as a guidance to infer the depth information from the 2D landmarks. Various artistic styles can also be applied to the generated face similar to how an artist would apply their own artistic style to a drawing. The controlled environment evaluations show that the reconstructed 3D faces are highly similar to the ground-truth examples. This approach can be used in many face modeling applications such as in 3D avatar creations, artistic face modelings, and police investigations.