F3FAD: Fast 3D Facial Avatar Digitization With XR Cloud Service
Qing Yang, Hua Zhang · 2022
As 3D Facial Avatar Digitization has become an increasingly important and popular feature for Extended Reality (XR), a high-fidelity and specially customized face avatar can greatly improve immersive experience for users who have been engaged in the virtual space. In this paper, we propose Fast 3D Facial Avatar Digitization (F3FAD), a novel end-to-end deep learning method for digitizing highly detailed 3D faces from a single image. There are four advantages of the F3F$A$D. Unlike previous works, F3F$A$D can infer both 3D mesh surface and color texture for overall human head model from a single view. There would be no need for sophisticated 3D scanning devices, multi-view stereo algorithms, or tedious capture procedures of hand-hold devices. It merely takes a single picture to digitize an entire face in the 3D world. Next, face attributes of the avatar can be freely manipulated according to user's preference. Thirdly, the avatar can be physically driven by multiple modalities of message inputs. At last, we have designed a novel AI computing framework to intelligently arrange the workload of F3FAD pipeline on a server-client system. In addition, we demonstrate the F3FAD technology with a speech-driven talking avatar, to showcase the advantages of our technology and system design in the real-world scenario.