A Framework for Multi-plane Image Layer Merging
Zachary McBride Lazri, Guan‐Ming Su, Peng Yin · 2023
The multi-plane image (MPI) is a promising framework for enabling novel view synthesis for 3D scene reconstruction. However, inefficient utilization of the MPI layers traditionally causes low compression efficiency, a large memory footprint, and high computational complexity. To address this issue, this paper proposes a framework for decreasing the number of layers in an MPI while producing high quality rendered images. We introduce a principled layer merging algorithm to reduce the number of layers in an MPI by using a constrained Lloyd-Max optimization formulation for determining how the layers should be merged. Since an imperfect MPI generation model may produce undesired artifacts in rendered images, which may be exacerbated after layer merging has been applied, a novel enhancement algorithm is constructed and applied before and after layer merging to remove artifacts and ensure the high quality of the layer-merged MPIs. Experimental results verify the utility of our framework, showing that: (1) our enhancement algorithm can significantly improve the quality of the MPI generated data, with an average increase in peak signal-to-noise ratio (PSNR) of $28 \sim 29$ dB and (2) our layer merging algorithm can successfully reduce the number layers in an MPI while producing high quality rendered images.