Multi-Scale Hierarchical VQ-VAEs for Blind Image Inpainting
Cheng Li, Hao Zhang, Dan Xu · 2024
Blind image inpainting aims to repair damaged parts of the image without prior knowledge. It is a challenging problem due to difficult to infer the background of the damaged area. Existing methods still produce unwanted outputs sometimes, with inconsistent or oversimplified details. To restore textures at a fine-grained level in blind image inpainting, we propose a model based on multi-scale hierarchical vector quantized variational autoencoders (VQ-VAE). It was firstly trained with a multi-scale hierarchical VQ-VAE module to learn efficient encoding of ground truth images to get 2 codebooks guiding the inpainting. Then another multi-scale hierarchical VQ-VAE module encodes corrupted images, and tries to align them to the codebooks. Finally, we added a decoder as well as the credibility mechanism to the second module to improve the inpainting quality. Experiments on several publicly datasets, including CelebA, Places2 and Paris StreetView, show that our method outperforms the current excellent techniques.