Augmenting depth estimation from deep convolutional neural network using multi-spectral photometric stereo
Yisong Luo, Hengchao Jiao, Lin Qi, Junyu Dong, Shu Zhang, Hui Ling Yu · 2017
Multi-Spectral Photometric Stereo can recover surface normals from a single image, but requires an initial estimate of the normals due to the tangle of the illumination, reflectance and camera responses on each of the RGB channels. Instead of employing a depth sensor or binocular stereo device, in this paper, we propose a method to estimate fine-scale geometry structures with the popular Deep Convolutional Neural Networks (CNNs). We train the network with rendered images of synthetic 3D objects, and apply the trained model with real world data. The CNN is used to estimate a rough prediction of the depth, then the normals from Multi-Spectral Photometric Stereo are progressively refined accordingly. Experiments demonstrate the competitive results of our method for improving the depth estimation.