Hole-filling for single-view plus-depth based rendering with temporal texture synthesis
D. M. Motiur Rahaman, Manoranjan Paul · 2016
View synthesis technique for 3D video and free viewpoint video (FVV) using existing view(s) can avoid the large volume of video data transmission. Existing techniques may concern poor rendering quality by missing pixel values (i.e. creating holes) due to the occluded region, rounding error and disparity discontinuity. To address those problems with the existing techniques uses correlations in spatial texture only or both spatial texture and temporal background. The former techniques (e.g. inpainting) suffer quality degradation due to lack of spatial correlation on the foreground-background boundary areas. The latter techniques (e.g. background update with Gaussian Mixture-based Modelling (GMM)) can improve quality in some occluded areas, however, due to the dependency on warping of background image and spatial correlation they still suffer quality degradation. In this paper, we propose a new hole-filling technique using the number of GMM model rather than the background image to identify background/foreground pixels. The missing pixels of background and foreground are recovered from the background pixel and the weighted average of warped and foreground model pixels respectively. The experimental results show that the proposed approach provides 0.9~1.7dB PSNR improvement compare to the state-of-the-art method.