Learning Fine-Scaled Depth Maps from Single RGB Images.
Jun Li, Reinhard Klein, Angela Yao · arXiv (Cornell University) · 2016
Inferring the underlying depth map from a single image is an ill-posed and inherently ambiguous problem. State-of-the-art deep learning methods can now estimate accurate depth maps, but when projected into 3D, still lack local detail and are often highly distorted. We propose a multi-scale convolution neural network to learn from single RGB images fine-scaled depth maps that result in realistic 3D reconstructions. To encourage spatial coherency, we introduce spatial coordinate feature maps and a local relative depth constraint. In our network, the three scales are closely integrated with skip fusion layers, making it highly efficient to train with large-scale data. Experiments on the NYU Depth v2 dataset shows that our depth predictions are not only competitive with state-of-the-art but also leading to 3D reconstructions that are accurate and rich with detail.