360 Panorama Synthesis from a Sparse Set of Images on a Low-Power Device
Julius Surya Sumantri, In Kyu Park · IEEE Transactions on Computational Imaging · 2020
A full 360o× 180oimage provides an unlimited field of view (FOV) and an immersive experience for the users without any loss of information of the surrounding. In this study, a deep learning based approach is proposed to synthesize a 360oimage from a sparse set of images captured with a limited FOV. The proposed network consists of a cascade of the FOV estimation network and the panorama synthesis network. We propose a hierarchical generative network to synthesize high quality 360opanorama images. The design of progressive multi-scale generator and multiple discriminator reduces the high frequency artifact which is commonly observed in image synthesis using generative networks. The network is further compressed to run on a low-power device such as a smartphone using our proposed size and latency optimization. Experimental result demonstrates that the proposed method produces 360opanorama with satisfactory image quality of up to 512 × 1024 resolution. It is also shown that the proposed method outperforms the alternative method and can be generalized for non-panoramic scenes and real images captured by a smartphone camera.