Blind Spectral Super-Resolution by Estimating Spectral Degradation Between Unpaired Images
Jie Xie, Leyuan Fang, Cheng Feng Wu, Feng Xie, Jocelyn Chanussot · IEEE Transactions on Geoscience and Remote Sensing · 2024
The spectral super-resolution (SpeSR) from multispectral images (MSIs) to hyperspectral images (HSIs) can bring rich spectral information. The deep learning-based methods have demonstrated their powerful ability for the SpeSR task, which requires the paired HSI/MSI to train the model. However, HSIs and MSIs are always obtained at different times and under different imaging conditions, covering different areas. To address this issue, in this paper, a framework named BliEstGAN based on the generative adversarial network (GAN) is proposed to estimate the spectral resolution degradation between unpaired HSIs and MSIs that can be used for the blind SpeSR. Specifically, each MSI imaging sensor has its own unique spectral sampling process, which can be modeled as a spectral degradation from its paired HSI. Different spectral degradations can be discriminated by the deep model. Therefore, the generator of the GAN is used to estimate the spectral degradation from HSIs to MSIs, and the discriminator of the GAN is adopted to distinguish whether the estimated and real spectral degradation are similar. The large difference in spatial resolution between MSIs and HSIs makes them easy to discriminate against. Therefore, smooth hyperspectral and multispectral patches are extracted from HSIs and MSIs to eliminate this difference in spatial resolution. Furthermore, according to the imaging sensor mechanism, some special regularization terms are designed for the generator to guarantee its correct convergence. Finally, the estimated spectral resolution degradation can be adopted to generate HSI/MSI pairs for the supervised learning-based SpeSR methods. Experimental results demonstrate the effectiveness of the proposed method.