Confidence-Driven Unimodal Interference Removal for Enhanced Multimodal Object Detection
Yu Wang, Shikui Wei, Sen Xu, Ying Hua Qin, Yao Zhao · IEEE Transactions on Circuits and Systems for Video Technology · 2025
Multi-spectral imaging senses objects from different perspectives, exhibiting the advantages of cross-modal collaboration. However, most existing cross-modal detection algorithms focus mainly on the design of fusion mechanisms, neglecting to assess the effectiveness of individual modalities. In fact, if a certain modality fails to provide distinguishable features, it will introduce unimodal interference and weaken the feature representation of dominate modalities. To address this problem, we propose an enhanced multi-spectral object detection algorithm via Confidence-driven unimodal Interference Removal (CIRDet). Specifically, we explicitly decompose unimodal visual contents into cross-modal consensus features and conflict features. For visual contents where both modalities express confidence, we employ an equal weighting fusion strategy to exploit the synergistic effect of modal information. In cases of modal discrepancy, we introduce the global and local feature confidence fusion mechanisms to induce the network to follow the guidance of the dominant modality, thereby removing conflicted interference from inferior modality. By decoupling the features and processing separately, the proposed method prevents the loss of valid information in inferior modality and filters out unimodal interference more accurately. Extensive experiments on three widely-used multi-spectral object detection benchmarks demonstrate our method outperforms state-of-the-arts by a large margin, e.g., with a CNN backbone, CIRDet achieves 4.2 mAP@[0.5:0.95] improvement compared to Transformer-based methods. The code will be released after possible publication.