Sketch-to-image Synthesis with Image-level Encoding Aggregation and Patch-level Semantic Refinement
Keqian Chen, Yonghua Zhu, Zhuo Bi, Wenjun Zhang · 2023
Sketch-to-image synthesis is a hotspot in computer vision, which can be applied to criminal investigation and recreation industry. To better satisfy the requirements of users, many researchers concentrate on the reference-based sketch-to-image task which aims to convert a sketch-like image into a photo-like image with a given appearance reference image. However, existing methods tend to produce broken geometry and incomplete appearance. In this paper, we develop a novel network with image-level encoding aggregation and patch-level semantic refinement to generate high-quality images from sketches with reference. The image-level part is developed with an aggregated encoder and serves as a feature amplifier to extract detailed features and enhance geometry reservation. And the patch-level semantic part works with the CIELAB space and scan line seed-filling algorithm to focus on local appearance transfer. Extensive results demonstrate significant enhancements in both geometry and appearance.