FCPENet: Feature Complementary Property Enhancement Network for RGB-Thermal Semantic Segmentation
Qingwang Wang, Haochen Song, Cheng Yin, Qingbo Wang, Hua Ren Wu, Tao Shen · 2023
RGB-thermal (RGB-T) semantic segmentation models have shown their effectiveness in achieving robustness under a variety of illumination conditions by leveraging the complementary information from RGB and thermal images. Despite significant progress in this field, existing RGB-T semantic segmentation methods often overlook the issue of information redundancy in multimodal learning. This oversight weakens the complementary property of cross-modal features, which contradicts the fundamental purpose of multimodal learning. To address this challenge, we propose a novel approach called the Feature Complementary Property Enhancement Network (FCPENet) for RGB-T semantic segmentation. We introduce a Feature Complementary Property Enhancement Module (FCPEM) that tackles the problem of redundancy between two modal features. The FCPEM employs a mutual information minimization constraint to effectively reduce redundancy and enhance the complementary nature of these features. Additionally, a channel attention mechanism is incorporated in the FCPEM to dynamically select critical complementary multi-modality features for further fusion. Benefited from the proposed FCPEM, our FCPENet enhances the complementary property of multimodal features and achieves improved segmentation performance. Extensive experiments conducted on the MFN dataset demonstrate the superiority of our FCPENet over state-of-the-art (SOTA) methods. Both subjective visual comparisons and objective metrics reveal the effectiveness of our approach, with an mIoU score that surpasses the top-ranked method on the current competition list by 0.5%.