Boosting Light Field Image Super Resolution Learnt From Single-Image Prior
Xingzheng Wang, Zixuan Wang, Wenhao Huang, Kaiqiang Chen, Lihua Li · IEEE Transactions on Computational Imaging · 2023
In recent years, many deep learning networks are proposed for light field super resolution (LFSR). LFSR problem is essentially ill-posed since unknown detail information need to be predicted. Hence LFSR networks require plentiful content information (e.g., shape, color, texture) learned from sufficiently diverse scenarios. However, due to the high collection cost, existing light field datasets are in small size and have few scenarios, which could not meet the requirement and limit the performance of LFSR networks. To solve this problem, we proposed a novel framework to significantly boost their performance. Our main idea is to introduce valuable and plentiful content information from single images into LFSR networks as prior. Specifically, first, a view synthesis method is applied to add unreal disparity into single images, increasing the dimensionality of single images, hence to solve the problem of inconsistent data modalities. Then, we design the Scenarios-Content Introduction Module (SCIM) to effectively extract content feature from synthesized data. Finally, due to the added unreal disparity in the first stage, the features have severely pseudo information. Hence the Feature Attention Module (FAM) is proposed to discriminately select valuable information and combine this information into the LFSR network. Extensive experiments on six datasets validate the effectiveness of the proposed method, leading to a maximum gain of 0.439 dB. Our method can even boost SOTA networks to achieve higher performance