ConsfomerST: Multilayer Transformer and Contrast Learning for Image Style Transfer
Yuanfeng Zheng, Honggang Zhao · 2024
In the task of image style transfer, accurately capturing style features is crucial. Current methods typically face two main issues: (1) mainstream methods often fail to capture fine-grained features in the image space and tend to lose details, resulting in images with noticeable artificial artifacts. Moreover, the style features extracted from a single image by neural network models are often underutilized, causing local distortions. (2) Due to the localized nature of convolutional neural networks (CNNs), extracting and preserving the global information of the input image becomes challenging. Traditional image style transfer methods suffer from biased content representation. To address these challenges, we propose ConsfomerST, a method based on transformers and augmented contrastive learning that directly learns style representations from large image datasets. This method considers long-range dependencies in input images during the style transfer process. Specifically, we utilize a transformer to process the style and content images into a specific sequence of generated images, followed by optimizing the model using an adaptive augmented contrastive learning method. We further introduce specialized negative sample libraries and sample penalization mechanisms to enhance model training. Qualitative and quantitative evaluations demonstrate that our method outperforms current state-of-the-art techniques.