Complementary Information Mining for Redundancy and Weakly Aligned RGB-T Semantic Segmentation
Qingwang Wang, Junlin Ouyang, Tao Shen, Yanfeng Gu, Sami Ullah, Mohammad T. Al-Antary, Hisham Alasmary, Muhammad Mohsin Waqas, Bo‐Hui Tang, Zahid Halim · IEEE Transactions on Geoscience and Remote Sensing · 2026
Multimodal data can effectively improve the accuracy and robustness of traditional RGB image semantic segmentation. However, the redundant information between cross-modal data hinders the complementary information mining of each modality. And the data misalignment between modes will aggravate the above effects. In this paper, we propose a complementary information mining network (CIMNet) for RGB-Thermal (RGB-T) semantic segmentation. We comprehensively consider the link between the difficulty of redundant information mining and modality misalignment. Through mutual information minimization and adaptive update of modality bias, we achieve more accurate and robust segmentation performance in complex environments. Specifically, we introduce a complementary information promotion and amplification (CIPA) module via mutual information minimization and channel attention mechanism to prevent a multi-modality network from focusing on redundant information and amplify the informative cross-modality features. Then, we design a spatial-channel sequential feature rectification (SCSFR) module with adaptive offset modeling to calibrate the modality misalignment features. Extensive experiments on public datasets demonstrate that our CIMNet outperforms other state-of-the-art (SOTA) methods in terms of objective metrics and subjective visual comparisons. The code will be accessible at https://github.com/KustTeamWQW/CIMNet.