Neural Style Transfer of Text Supervision
Bin Xie, Xiang Shao · 2023
Style transfer technology has been widely applied in the field of image processing. Most of the current style transfer methods obtain style information from a single mode. This will cause the model to lose or incomplete information during style transfer, which will affect the visual effect of the resulting image. Multimodal technology is a method that combines the information of different modes together to realize a more comprehensive understanding and processing of data. It has the advantages of rich information expression, cross-modal interconnection, complementing and complementing each other, which provides a new idea for style transfer method. Based on CNN, this paper proposes a style transfer method using text information as constraint. On the one hand, the style information provided by the text makes up for the deficiency of the visual modal information, which helps the model to learn more style information. On the other hand, the semantic structure and spatial relationship provided by the text playa constraining role in maintaining the semantic structure of the content image. The experiment shows that compared with the traditional style transfer method, the style transfer method with text constraint can solve the problem of information loss and incomplete by using a single mode style transfer, and get better experimental results.