Adaptive Hyperspectral Siamese Network in Transformer
Chang Liu, Jiawei Zhou, Yanni Dong · 2023
Hyperspectral object tracking is a technique that involves real-time or near-real-time monitoring and localization of specific object using hyperspectral data. In this technology, the reflective spectral information of objects in different bands is utilized to extract target features, enabling tracking and surveillance. Hyperspectral data contains more spectral information data than RGB images, and in hyperspectral images, the categories are more differentiated, which can be utilized in object tracking to target with higher accuracy. To address this feature, we design an end-to-end Adaptive Hyperspectral Siamese network (AHS-Net). First, we build an adaptive feature extraction transformer backbone that can simultaneously process hyperspectral data with false color data. Second, in order to enable the hyperspectral data to share the feature extraction backbone with the RGB data, we designed a band adaptive selector, a module that utilizes the spectral and spatial attention mechanism to sort different bands according to their informativeness, and then form three-channel groups. Finally, we use the multi-branch network to achieve the fusion of classification results of fused hyperspectral data and false-color data to achieve the final tracking mission.