Multi-Modal Hybrid Interaction Vision-Language Tracking
Lei Lei, Xianxian Li · IEEE Transactions on Multimedia · 2025
Vision-language tracking is a crucial branch of multi-modal object tracking, aiming to jointly locate an object by utilizing visual information and language descriptions. Typically, existing vision-language trackers employ language and visual encoders to extract features from language descriptions and visual information, respectively. Based on these extracted visual and language features, a cross-modal interaction module is used to extract multi-modal features to locate the targets. However, they ignore the differences between visual and language modalities. Due to the lack of pixel-level position information in language descriptions, the positional information of the multi-modal features is greatly weakened by the cross-modal interaction modules. As a result, the vision-language trackers cannot effectively capture subtle changes in the target's positions. To address this problem, we propose a multi-modal hybrid interaction vision-language tracking method (named MHITrack), in which a multi-modal hybrid interaction decoder is designed to enhance the positional information of multi-modal features. The proposed multi-modal hybrid interaction decoder consists of a visual-language interaction module, a multi-level position interaction module, and a hybrid interaction module. Firstly, the multi-level position interaction module is utilized to capture fine-grained position information of the target from multi-level features. Meanwhile, the visual-language interaction module performs cross-modal interaction between visual and language features to obtain multi-modal features. Furthermore, the hybrid interaction module is employed to integrate the multi-modal features with target position information, enhancing the positional information of the multi-modal features. Finally, the proposed tracker can effectively capture subtle changes in the target's positions. Through extensive experiments on four benchmark datasets, namely TNL2k, LaSOT, OTB-Lang, and LaSOText, we demonstrate that the proposed vision-language tracker achieves promising performance compared to existing state-of-the-art vision-language trackers.