Three-stage progressive modality reweighting and interaction framework for robust RGB-T tracking
He Li, Jia Liang, Weihang Kong, Xingchen Zhang · Knowledge-Based Systems · 2026
RGB and thermal infrared (TIR) modalities provide complementary visual cues, enabling more robust tracking in adverse lighting and weather conditions. However, most existing RGB-T trackers focus on static feature aggregation, lacking awareness of how modality reliability evolves over time and across scenes. This often leads to unstable performance when the visibility of one modality fluctuates or degrades. To address this issue, we propose a Three-stage Progressive Modality Reweighting and Interaction (TPMRI) framework that progressively models temporal dynamics, semantic interaction, and global fusion consistency. Specifically, a temporal modality reweighting strategy captures short-term temporal variations and dynamically adjusts modality reliability; a cross-modality interaction mechanism establishes fine-grained semantic alignment between RGB and TIR features, facilitating TIR-guided information compensation; and a modality-aware fusion stage performs adaptive global aggregation for stable and consistent representations. These three modules are integrated into the core component, namely the Three-stage Progressive Interaction (TPI) structure, which unifies temporal, semantic, and global modeling within a single framework. By incorporating TPI into the tracker, TPMRI effectively mitigates temporal bias accumulation and enhances robustness under dynamic conditions. Extensive experiments on GTOT, RGBT234, and LasHeR benchmarks demonstrate that our method achieves superior tracking accuracy and success rate compared with state-of-the-art RGB-T trackers.