Stereo and Mono Depth Estimation Fusion for an Improved and Fault Tolerant 3D Reconstruction
Mircea Paul Muresan, Marchis Raul, Sergiu Nedevschi, Radu Gabriel Danescu · 2021
Depth estimation approaches are crucial for environment perception in applications like autonomous driving or driving assistance systems. Solutions using cameras have always been preferred to other depth estimation methods, due to low sensor prices and their ability to extract rich semantic information from the scene. Monocular depth estimation algorithms using CNNs may fail to reconstruct due to unknown geometric properties of certain objects or scenes, which may not be present during the training stage. Furthermore, stereo reconstruction methods, may also fail to reconstruct some regions for various other reasons, like repetitive surfaces, untextured areas or solar flares to name a few. To mitigate the reconstruction issues that may appear, in this paper we propose two refinement approaches that eliminate regions which are not correctly reconstructed. Moreover, we propose an original architecture for combining the mono and stereo results in order to obtain improved disparity maps. The proposed solution is designed to be fault tolerant such that if an image is not correctly acquired or is corrupted, the system is still able to reconstruct the environment. The proposed approach has been tested on the KITTI dataset in order to illustrate its performance.