Image Inpainting with Information Loss Reduction and Texture-Structure Feature Fusion
Fang Long, Yuan‐Gen Wang · 2023
Image inpainting has made great progress with the help of deep learning. However, existing methods show performance degradation when restoring corrupted images with complex scenes. In this paper, we propose a novel image inpainting method by reducing intermediate layer information loss and fusing texture-structure features. To be specific, we first compute a Local Binary Pattern (LBP) map of the corrupted image as the input of structure feature extraction, considering that LBP contains richer structure information than edges and contours. Then, we introduce a Wide Identical Residual Weighting (WIRW) module to utilize the intermediate layer features in the structure encoder. Furthermore, we introduce a Spatial-Transformer (ST) module consisting of Convolutional Neural Network (CNN) and Transformer branches to fuse the structure and texture features, where the CNN and Transformer branches are responsible for capturing the local and global information, respectively. Various experiments on public datasets including CelebA, Paris StreetView, and Places2 demonstrate the effectiveness of the proposed method. Especially, our ablation study separately verifies the contribution of each module to the whole framework. The code is publicly available at https://github.com/GZHU-DVL/LFang.