Multi-Spectral Pedestrian Detection Guided by Feature Consistency
Keyu Zhao, Chengyue Qian, Jianyang Fang, Yong Wang, Lu Ding · 2023
Multi-spectral pedestrian detection has been proven to overcome the limitations of visible-modal pedestrian detection by using both visible and thermal images, such as low light, cluttered background, thus enabling all-day pedestrian detection. However, the fusion strategy, which directly uses feature addition, cannot make full use of multi-spectral image information, and even interferes with the detection results. In this paper, the feature consistency module is introduced to guide the features of different modalities to map into the same space, so as to minimize the differences of consistent semantic feature information between different modalities. In addition, we introduce the foreground confidence weight perception module to quantify the importance of two modalities for detection results and fuse the feature by weight, so as to make full use of the features of different modalities. The results on Kaist dataset and LLVIP dataset show that the improved method can alleviate differences of modality in image pairs. It achieves 8.48 % and 2.82 % missing rate on Kaist dataset and LLVIP dataset. Compared with SOTA results, these experiments can demonstrate the effectiveness and generalization of our method.