DMPD: A Dual-Modality Fusion Method for Cross-Spectral Pedestrian Detection
Huanyu Yang, Jun Wang, Mengchu Tian, Yuming Bo · IEEE Transactions on Human-Machine Systems · 2025
In urban safety, intelligent transportation, and smart security applications, robust pedestrian detection is paramount. Methods that rely solely on visible light imaging struggle in low-light or adverse weather conditions. To address these challenges, we propose dual-modality pedestrian detection (DMPD)—a novel dual-modality pedestrian detection framework that fuses visible and infrared imaging through innovative fusion strategies. The method integrates a modal alignment module to reduce pixel-level misalignment, a differential modal fusion module to effectively combine complementary features while suppressing noise, and a mix module that enhances multiscale feature extraction via integrated convolution and self-attention mechanisms. Furthermore, the enhanced YOLOv7 is used to further boost feature representation and detection accuracy. Experimental results on the public dataset demonstrate that DMPD achieves a detection$mA{{P}_{50}}$of 97.1% and a real-time speed of 118 FPS, outperforming state-of-the-art methods under both normal and adverse conditions, including fog, rain, and snow. These results confirm the effectiveness of the proposed fusion strategy in harnessing the complementary strengths of visible and infrared modalities, thereby offering a highly robust and scalable solution for pedestrian detection in complex urban environments.