A Multi-Stream Visual-Spectral-Spatial Adaptive Hyperspectral Object Tracking
Pengfei Wei, Qiao Liu, Zhenyu He, Di Yuan · 2025
Hyperspectral videos contain rich spectral and spatial information, which has enormous potential for object tracking compared to traditional RGB videos. However, the limited hyperspectral data, spectral differences across different bands, and high computational costs result in existing trackers being unable to effectively build connections between spectral and spatial information, leading to suboptimal tracking performance. To address these issues, this paper proposes a multi-stream Visual-Spectral-Spatial adaptive hyperspectral object tracking model(VSS). First, we design a multi-stream hyperspectral Transformer module to receive spectral data from different bands and extract Visual-Spectral-Spatial features across those bands. Then, to address the band differences between different modalities, we introduce the Bidirectional Visual-Spectral-Spatial Adapter module, which adaptively fuses visual, spectral, and spatial information from different bands. Finally, experiments on the HOTC dataset demonstrate the excellent performance of the VSS model.