Multi-modal Image Super-resolution with Joint Coupled Deep Transform Learning
R Krishna Kanth, Andrew Gigie, Kriti Kumar, A Anil Kumar, Angshul Majumdar, P. Balamuralidhar · 2022 30th European Signal Processing Conference (EUSIPCO) · 2022
In this paper, we address the problem of multi-modal image super-resolution (MISR), which aims at improving the resolution of the target modality with the help of high resolution guidance image of another modality. A novel joint coupled deep transform learning framework (JCDTL) based on deep transform learning is proposed which combines the information from multiple modalities for achieving MISR. The formulation and the requisite solution steps are provided. Two publicly available datasets RGB/NIR and RGB/Multispectral are considered for performance evaluation. The proposed approach shows a considerable improvement in performance compared to the state-of-art techniques. Further, an average PSNR improvement of close to 2dB and 1.5dB on RGB/Multispectral and RGB/NIR datasets respectively is observed by increasing the number of layers from one to three.