Partial style transfer using weakly supervised semantic segmentation
Shin Matsuo, Wataru Shimoda, Keiji Yanai · 2017
In this paper, we propose a partial texture style transfer method by combining a neural style transfer method with segmentation. A style transfer technique based on Convolutional Neural Network (CNN) can change appearance of an image naturally while keeping its structure. We extend this algorithm for changing appearance partly in a given image. For example, changing a ball made of “leather” in the image to one made of “metal”. The original algorithm changes the style of an entire image including the style of background even though we want to change only object regions. Therefore, we need information of target object position, in order to transfer texture styles to only object region in an image. We segment target object regions using a weakly supervised segmentation method and transfer a given texture style to only the segmented regions. As results, we achieved partial style transfer for only specific object regions, which enables us to change materials of objects in a given image as we like.