Deep Siamese Multi-level Feature Network for VIS and NIR Image Matching
Xinyi Ma, Fan Wang · 2023
Infrared (NIR) and visible light (VIS) images matching is a critical issue in the field of computer vision, aiming to align and correlate images from different spectral ranges. The structural deformation between NIR and VIS light images, alongside nonlinear radiation differences, results in lower generality and efficiency of existing image matching methods. Addressing the inadequacy of existing methods in providing the necessary high-level feature information differences for matching cross-modal images, we propose a deep Siamese multi-feature network for matching VIS and NIR images. Firstly, we propose multi-tier Siamese sub-network components for recursive inference, focusing on understanding high-level image features to obtain more spatially rich semantic information. Subsequently, we employ Transformer encoders to attend to multi-scale Siamese CNN's multi-level feature maps to decode complex nonlinear feature differences between cross-modal images caused by different physical processes. Finally, experimental results on multimodal datasets demonstrate that our method outperforms the SOTA methods in matching metrics.