Semi-Supervised Landmark-Guided Restoration of Atmospheric Turbulent Images
Chun Pong Lau, Amit Kumar, Rama Chellappa · IEEE Journal of Selected Topics in Signal Processing · 2021
Image degradation due to atmospheric turbulence (AT), which is common while capturing images at long ranges, adversely affects the performance of tasks such as face alignment and face recognition. To the best of our knowledge, there does not exist any dataset consisting of turbulence-degraded face images along with their annotated landmarks and ground-truth clean images, making supervised training challenging. In this paper, we present a semisupervised method for jointly extracting facial landmarks and restoring the degraded images by exploiting the semantic information from the landmarks. The proposed approach learns to generate AT images by combining the content from a clean image and turbulence information from AT images in an unpaired manner. Next, we use heatmaps from the landmark localization network as a prior to the image restoration module. Subsequently, we impose heatmap consistency loss and heatmap confidence loss to regularize the restored images. Extensive experiments demonstrate the effectiveness of the proposed network, which achieves an NME of 2.797 on the task of landmark localization for strong turbulent images and yields improved restoration results compared to state-of-the-art methods.