Light Field Editing Propagation using 4D Convolutional Neural Networks
Zhicheng Lu, Xiaoming Chen, Yuk Ying Chung, Zhibo Chen · 2020 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW) · 2020
2D image editing has been a well-studied problem. However, 2D image processing techniques cannot be directly applied to the emerging light field image (LFI) due to the particular structural characteristics of LFI. Without a dedicatedly designed editing scheme for LFI, users need to manually edit each sub-view of the LFI. This process is extremely time consuming, and more importantly, users have no ways to guarantee parallax consistency between sub-views. This poster proposes two different LFI editing schemes including the direct editing scheme and the deep-learning-based scheme. These schemes enable automatic propagation of the user’s edits, particularly “augmentation” editing, from central view to all the other sub-views of LFI. In particular, the learning-based scheme employs interleaved spatial-angular convolutions (4D CNN) to enable effective learning of both spatial and angular features, which are subsequently used to help the augmentation editing. We constructed a preliminary LFI dataset and compared the proposed two schemes. The experimental results show that the learning-based scheme produces higher PSNR (0.51dB) and more pleasant subjective editing results than the direct editing.