ASFN: An RGB-T Adaptive Selection Fusion Network for Nighttime Tracking
Linxin Zhang, Xilin Chen, Jinsheng Liu, Tao Yan · 2024
Existing state-of-the-art tracking methods always focus on conditions with good illumination, but neglect the adverse effects caused by dark nights. The low visibility at nighttime makes it is difficult to distinguish the target from the background in cases of insufficient illumination. Even state-of-the-art trackers are applied, it is still hard to achieve detection performance comparable to that in the daytime. In order to overcome the above challenges, we propose a novel network for nighttime tracking, called RGB-T (Thermal) Adaptive Selection Fusion Network (ASFN), which takes multi-modal data, i.e., RGB-T, as input. Specifically, to make use of the complementary and discriminative information of the two types of modal images, we propose an adaptive selection fusion strategy. In addition, for alleviating the problem of parallax between different modalities in RGB-T datasets, we propose an asymmetric local window cross attention fusion mechanism (ALW-MCA), which could integrate features by dividing them into asymmetric local windows and introducing cross attention. For learning our proposed network, we also use methods of simulating nighttime data, by adopting gamma correction and image darkening to darken the daytime videos of RGB-T datasets. Extensive experiments conducted on real-world and synthetic nighttime video data demonstrate the effectiveness and superiority of our method.