OctNet: Illumination-Aware Octave Fusion and Feature Enhancement for Multispectral Pedestrian Detection
Sirui Wang, Guiling Sun, Liang Dong, Bowen Zheng · IEEE Sensors Journal · 2025
Multispectral pedestrian detection, which combines visible and infrared images, has demonstrated significant advantages under various lighting and weather conditions, making it a highly focused research topic in recent years. The RGB-T modality essentially provides different descriptions of the same scene, encompassing both modality-specific and modality-consistent information. However, most existing approaches overlook the differences between these two types of information during feature fusion, leading to insufficient feature representation. To address this, we propose an illumination-aware octave fusion framework (OctNet) for RGB-T pedestrian detection. Specifically, we introduce an illumination-guided Octave Fusion module (IA-OctFuse), which utilizes frequency domain analysis to separate modality-complementary target features from redundant background features. Additionally, an illumination-aware mechanism is incorporated to adaptively balance the contributions of different modalities, producing highly discriminative RGB-T fused features. Then, a Multi-Head Dilated-convolution module (MHCE) is designed to deeply explore the spatial self-similarity of fused features, further enhancing feature representation. Extensive experiments and comparisons show that the proposed OctNet achieves state-of-the-art performance on publicly available KAIST and LLVIP pedestrian detection datasets.