Unsupervised 3D Face Reconstruction with Reprogramming Skip Connections
Zhuoming Dong, Huajun Zhou, Jianhuang Lai · 2023
In unsupervised 3D face reconstruction, existing methods modeling the canonical face typically exclude the skip connections between encoder-decoder pairs. Consequently, they have difficulty capturing appearance details necessary for the task. However, directly applying original skip connections only induces these methods to degrade to a trivial 2D texture reconstruction algorithm. In this paper, we propose novel Reprogramming Skip Connections (RSCs), which escape from bringing about degradation and improve the 3D face reconstruction quality. Specifically, the proposed method filters out inappropriate information causing degradation by aggregating the features from the encoder in spatial dimensions into several prototypes. These prototypes preserving beneficial information are subsequently combined with the corresponding decoder features with the help of expansion masks. Further, we design the masks reconstruction consistency loss to improve the quality of the expansion masks. Our experiments verify the superiority of our method compared to other competitors.