CReStyler: Text-Guided Single Image Style Transfer Method Based on CNN and Restormer
Long Feng, Guohua Geng, Yong Ren, Zhen Li, Yangyang Liu, Kang Li · 2024
Text-guided image style transfer methods have gradually become a research hotspot. However, existing text-guided style transfer method suffers from content information missing and artifacts in the generated stylized images. Therefore, we propose CReStyler, a text-guided image style method based on the dual-branch structure of CNN and Restormer. In this work, in the first branch, we introduce a novel convolutional structure called FcasNet, which composes Frequency-domain Channel Attention Mechanism (FcaNet) and Cross Convolutional Block Attention Module (CRCBAM). It can generate rough style images lfcaaccording to the input target text. In the second branch, the use of pixel-based Restormer constrains the phenomenon of fake images and content information missing due to excessive convolution. It can generate stylized images lreswith complete content information based on the target text. Finally, we combine lfcaand lresthrough weighted fusion to obtain refined stylized images. During the training process, we utilize directional CLIP loss to constrain text-image alignment. Experimental results show that our method produces better results compared with existing methods such as CLIPStyler, LDAST, Text2LIVE, InstructPix2Pix.