Image Guided Depth Super-Resolution for Spacewarp in XR Applications
Christopher Peri, Yingen Xiong · 2021
In this paper, an image guided depth super-resolution approach is presented for generating higher-quality and higher-resolution depth maps from lower-resolution depth points, such that it can be used in depth based image re-projection (a.k.a: Spacewarp) and other applications in XR. In this approach, a higher-resolution depth map image is reconstructed using spatial, intensity, and depth information of neighborhood pixels. Three kinds of weights are computed with neighborhood information. The first one uses 3D pose information, the second one uses color image intensity information, and the third one uses color image spatial information. We compute the weighted sum of the depths in neighborhood and accumulate the weights. The depth candidate for the current considered point is determined by the weighted average of the depths in neighborhood. We build a criterion to find an optimal solution for the current pixel. After all pixels are processed, a higher-resolution depth map is obtained.Applications and result analysis for the approach are also presented in this paper. With different levels of down sampled depth points, higher-resolution depth maps are generated with a generally used regular grid interpolation approach and the image guided depth super-resolution algorithm presented in this paper. After comparing and analyzing, we can see that our algorithm can generate a higher-resolution depth map with more accurate and clearer objects and boundaries than generally used interpolation approaches.