Few-Shot Facial Sketch Synthesis via Progressive Domain Gap Reduction
Dan Lu, Zhenxue Chen, Chengyun Liu, Yuchen Hu, Lei Cai, Q. M. Jonathan Wu · IEEE Transactions on Information Forensics and Security · 2025
Facial sketch synthesis (FSS) has advanced significantly in recent years, but challenges remain in few-shot settings. Some few-shot learning methods can convert photos (source domain) into sketches of a specified style (target sketch domain). However, they overlook the available samples of other sketch styles (non-target sketch domains). We argue that the information in these samples can help the model enhance its mapping ability from the source domain to the target domain. This paper proposes a progressive domain gap reduction (PDGR) method for few-shot facial sketch synthesis, which consists of three stages: teacher training, knowledge distillation, and intra-domain few-shot adaptation. In the first stage, we adapt a pretrained StyleGAN to a non-target sketch domain with more available samples than the target sketch domain. To generate diverse and high-quality sketches, we employ a dual-discriminator adversarial mechanism to guide the model in focusing on the overall structure and style, as well as multi-scale details and textures. In the second stage, the knowledge from StyleGAN is transferred to a U-Net for more efficient image translation. In the third stage, we adapt the output of the U-Net from the non-target sketch domain to the target sketch domain in few-shot settings. To alleviate overfitting, preserve individual characteristics, and enhance detail representation, we leverage the FFHQ dataset to construct dual training paths and design a domain-directional triple loss. Experiments show that PDGR significantly outperforms previous few-shot learning methods and even outperforms the state-of-the-art FSS methods trained on the full dataset.