Blind Face Restoration via Multi-head Cross-attention and Generative Priors
Zhiyong Huo, Shanlin Hu · 2023
In the real world, blind face restoration from face images with unknown degradation is a challenging problem. Directly training deep neural networks often fails to achieve reasonable results due to unknown severe degradations. Existing methods based on generative models produce good results but produce restoration results with overly smooth textures or unnatural overall structures. To address this, we propose a novel approach that combines the strengths of convolutional neural networks and Multi-head Cross-attention. Initially, we employ ResNet50 to generate preliminary W+ latent vectors and multi-resolution scale feature maps. We then enhance the multi-scale feature map using cross attention in a Transformer-based feature match module. This process improves the correlation between spatial features and intermediate semantic features of generativa priors, enhances the distance dependence among different semantic features, and extend latent space to improve semantic expressiveness. We also leverage pre-trained GANs to enhance image reconstruction and introduce an FFT-based frequency-domain loss function to pay more attention to high-frequency features. Experiments show that our model outperforms existing Blind Face Restoration methods in both objective metrics and subjective quality comparisons, especially when restoring severely degraded images in real-world scenarios, resulting in visually realistic results.