Unsupervised Depth and Bokeh Learning from Natural Images Using Aperture Rendering Generative Adversarial Networks
Takuhiro Kaneko · NTT technical review · 2022
Humans can estimate the depth and bokeh effects from a two-dimensional (2D) image on the basis of their experience and knowledge.However, computers have difficulty in doing this because they logically cannot have such experience and expertise.To overcome this limitation, a novel deep generative model called aperture rendering generative adversarial network (AR-GAN) is discussed.AR-GAN makes it possible to control the bokeh effects on the basis of the predicted depth by incorporating an optical constraint of a camera aperture into a GAN.During training, AR-GAN requires only standard 2D images (such as those on the web) and does not require 3D data such as depth and bokeh information.Therefore, it can alleviate the application boundaries that come from the difficulty in collecting 3D data.This technology is expected to enable the exploration of new possibilities in studies on 3D understanding.