Arbitrary-scale dense light field reconstruction with meta-learning
Sen Xiang, Yi Zhang, Huiping Deng, Jin Wu · 2023
Light field (LF) suffers from great data volume, and an effective approach to cope with this problem is to reconstruct dense LFs from sparse ones. However, the existing LF reconstruction methods are performed with given and fixed up-scaling factors and thus lost the flexibility and the ability of adapting to different applications. In this paper, we propose to reconstruct dense LF with meta-learning. The framework consists of two inputs, e.g. the sparse light field and the angular coordinate of the desired viewpoint, and three modules: feature extraction module that extract sparse LF features, weight prediction module that predicts view-adaptive weights, and view synthesis module that exactly generates virtual new views. Especially, different from conventional work, the proposed framework leverages meta-learning that learns view-adaptive synthesis parameters, so that it can be used to render arbitrary viewpoints and reconstruct dense LF with arbitrary up-scaling factors. The proposed framework is verified with extensive experiments, which demonstrate that it can generate arbitrary views and reconstruct dense LFs with high quality and high efficiency that outperforms the state-of-the-art methods.