Facial landmark disentangled network with variational autoencoder
Sen Liang, Zhize Zhou, Yudong Guo, Xuan Gao, Ju-yong Zhang, Hujun Bao · Applied mathematics/Applied Mathematics. A Journal of Chinese Universities/Gao-xiao yingyong shuxue xuebao · 2022
Abstract Learning disentangled representation of data is a key problem in deep learning. Specifically, disentangling 2D facial landmarks into different factors (e.g., identity and expression) is widely used in the applications of face reconstruction, face reenactment and talking head et al.. However, due to the sparsity of landmarks and the lack of accurate labels for the factors, it is hard to learn the disentangled representation of landmarks. To address these problem, we propose a simple and effective model named FLD-VAE to disentangle arbitrary facial landmarks into identity and expression latent representations, which is based on a Variational Autoencoder framework. Besides, we propose three invariant loss functions in both latent and data levels to constrain the invariance of representations during training stage. Moreover, we implement an identity preservation loss to further enhance the representation ability of identity factor. To the best of our knowledge, this is the first work to end-to-end disentangle identity and expression factors simultaneously from one single facial landmark.