Dynamic interaction transformer and refined feature alignment for enhanced multispectral pedestrian detection
Lujuan Deng, Xinglong Liu, Zuhe Li, Min Jiang, Fangmei Liu, Wei Wang · Journal of Electronic Imaging · 2025
Multispectral pedestrian detection, integrating vsible and infrared image data, enables accurate pedestrian identification and is crucial for applications such as autonomous driving and security surveillance. Existing convolutional neural network (CNN)-based fusion methods face limitations due to their fixed receptive fields, which restrict the capture of only local information. To address this, we propose DITNet, a framework utilizing a dynamic interaction transformer for multimodal feature fusion. This transformer aggregates contextual information from locally adjacent keys, associates the obtained key features with queries, and computes a weighted sum of attention maps and their corresponding values. This process naturally considers interactions between queries and locally adjacent keys, enabling the capture of long-range dependencies and global contextual information. Moreover, to mitigate target misalignment, we introduce a refined feature alignment module that integrates local and global information from multiscale features to predict and correct feature offsets, addressing information mismatch. Extensive experiments on the challenging Kaist and FLIR datasets demonstrate that DITNet significantly outperforms existing methods, underscoring its effectiveness and robustness. Our code is available at: https://github.com/12lele/DITNet?tab=readme-ov-file