Intensity Guided Depth Upsampling Using Edge Sparsity and Super-Weighted $L_0$ Gradient Minimization
Shengtao Yu, Hui Lan, Cheolkon Jung · IEEE Access · 2019
Although depth cameras acquire depth in dynamic scenes, the captured depth images are often noisy and of low resolution. Depth images have the physical nature of being represented by smooth regions and edges in between them, i.e. depth images have high edge sparsity in the gradient domain. In this paper, we propose intensity guided depth upsampling using edge sparsity and super-weighted L0gradient minimization. First, we get mutual structure between intensity and depth using joint mutual structure filtering. Second, we generate an initial depth image using recursive interpolation. Next, we generate weights for L0gradient minimization based on gradient and entropy of the intensity image, and upsample the depth image using super-weighted L0gradient minimization. In super-weighted L0gradient minimization, we combine two main terms: Hybrid data fidelity and weighted L0gradient regularization. The hybrid data fidelity term combines both zero-order and first-order data differences to suppress staircase artifacts, while the weighted L0gradient regularization term preserves depth structures and removes noise. Finally, we further refine the depth image using adaptive fast weighted median filtering. Experiments on Middlebury and realworld scene datasets verify that the proposed method produces edge preserving depth upsampling results and outperforms state-of-the-arts in terms of both visual quality and quantitative measurements.