Modified Neural Style Transfer using the Chan-Vese segmentation and the Laplacian Image Pyramid
Navdeep Singh Chaudhary, Meenu Gupta, Rakesh Kumar · 2023
In recent times, Neural Style Transfer has been a highly probed area of Machine learning. Style Transfer refers to the fashion of embedding the Style of one image onto another image while conserving its natural contents. As transferring the style of one image to the target image is an NP-hard problem, it is veritably delicate to achieve and largely computationally ferocious. Also, the Content and Style of an image are very idiomatic and hard to express in terms of fine equations. But with the help of the Gram Matrix, the insight into the style of an image is more or less expressed. The gram matrix is extracted from the feature maps taken from some formerly pre-trained models like VGG19, YoloV4 etc. The Total loss function is defined as a combination of the content loss function captured by the layers of the neural network and the style loss function is deduced from the attained Gram matrix. The basic idea of this paper is that the energy difference between the generated image and the original content image should not differ much because content must be preserved up to 70-80% or more, thus to lower the difference, the high-frequency content details are embedded in the generated image at every cycle or at some intervals. In this paper, the author has modified the process by introducing the high-frequency content details present at the minimal energy area of the content image so as to reduce the number of iterations required to converge and have better content details in the generated image as opposed to generic neural style transfer. The high-frequency component of the content image is derived from the Laplacian Image pyramid algorithm and the Chan-vese algorithm gives a lower energy area of the image segments.