Denoising-Based Decoupling-Contrastive Learning for Ubiquitous Synthetic Face Images
Yupeng Zhu, Xinyi Shen, Peilun Du · IEEE Access · 2023
With the improvement of generative models such as GPT-4, GANs, and diffusion models, synthetic face images are increasingly pervading the current digital environment. Various face editing software based on generative models is already commercially available, which can edit face image attributes, including changing age, makeup, hair, scars, gender, etc. Existing face recognition methods trend to employ synthetic face images to augment datasets during large-scale training. However, the involvement of low-quality synthetic data can impair the feature extraction ability, consequently affecting recognition performance. Furthermore, face editing can potentially be applied for illegal or criminal purposes, such as criminals uploading edited face images for disguising themselves, thereby reducing the accuracy of face recognition models. To mitigate the negative impact of synthetic face images, we propose a denoising-based decoupling-contrastive learning (DDCL) method for extracting more benign features from synthetic data. By designing a siamese network structure with two branches, the framework extracts robust features from natural and synthetic images with contrastive learning mechanism. Subsequently, the bi-directional coding-based feature decoupling module filters out features of synthetic images before proceeding to identity recognition. Experimental results demonstrate that our method can alleviate the negative impact of synthetic face images and achieves the highest recognition accuracy for both synthetic and natural data.