Point Cloud-Based Free Viewpoint Artistic Style Transfer
Eunjee Bae, Jae‐Kyung Kim, Sanghoon Lee · 2023
In recent years, artistic style transfer has gained popularity as a means of creating visually appealing images by injecting style into the content image. Although various methods have been proposed for 2D domain, style transfer in the 3D domain still faces numerous challenges that hinder its quality and efficiency despite the arising demand of 3D contents. Therefore, we propose a novel free viewpoint style transfer framework that utilizes a point cloud representation. By encoding neural image features to each point in point cloud, free viewpoint rendering is accomplished by rasterizing features to 2D plane and decoding them with a neural renderer. During this process, style features are injected to features in point cloud with attention-based adaptive instance normalization module enabling to obtain view-consistent stylization images reflecting the style image without additional training. Experimental results validate that our method generates superior inter-view consistent stylized images compared to other existing approaches in higher resolution. Additionally, we demonstrate the editability of our framework where the scene can be simply edited through simple modification of point cloud geometry.