Attentive Partial Convolution for RGBD Image Inpainting
Ankan Dash, Guiling Wang, Tao Han · 2024
In this work, we demonstrated the use of Partial Convolutions for RGBD image inpainting. We proposed the two models L-PConv and Attn-PConv. The baseline partial convolution model is outperformed by both of our proposed models, with the Attn-PConv model performing the best. The proposed Attn-PConv model is able to infill missing pixels with relatively less training time when compared to some other GAN-based models. As far as we know this is the first time a partial convolution model has been used successfully for RGBD image inpainting. The results( Image SSIM: 0.9787, Image PSNR: 30.9665, Depth SSIM: 0.9818, Depth PSNR: 35.7311) indicate that our model is successful in RGBD image inpainting. The addition of the additional loss terms and the Attentive Normalization techniques help improve the performance of the model significantly. We believe our model can be successfully used in AR-related applications where infilling missing pixels is performed frequently especially for both RGB and Depth images together. Beyond the scope of this study, we envision practical applications for our model in augmented reality, particularly in scenarios where frequent pixel infilling is required for both RGB and Depth images. In the future our research trajectory aims to incorporate higher resolution, aligning with the capabilities of modern cameras capable of capturing images at 4k resolution.