Mask Refined Deep Fusion Network With Dynamic Memory for Robust RGBT Tracking
Ce Bian, Sei‐ichiro Kamata · 2023
In the field of object t racking, single RGB-based trackers are insufficient to cope with various extreme environments. Therefore, using complementary bimodal inputs of RGB and thermal infrared information in RGBT tracking has become increasingly popular. However, common RGBT trackers still face challenges in improving the efficiency of bimodal utilization and long-term sequence tracking. This paper proposes a novel network that integrates spatial and temporal information to perform robust RGBT tracking. Specifically, a three-branch backbone network is designed to extract shallow fused features with low complexity. To address the problem of insufficient modal fusion and noise, a deep fusion module is designed to enhance the utilization of common and complementary information between modalities. In addition, a memory module is designed to use temporal information dynamically, reducing tracking failure when occlusions occur. Finally, we proposed a mask refinement module to obtain pixel-level boundary information, reducing the probability of bounding box drift and achieving more accurate scale estimation of the target. Experimental results show that our tracker achieves highly competitive results in terms of performance and speed on multiple RGBT tracking datasets.