Unsupervised Deep Infrared and Visible Homography Estimation Algorithm Based on Content-Aware
Y. Y. Liao, Yinhui Luo, Xingyi Wang · 2023
Homography estimation is a method that describes the geometric projection relationship between images. Traditional homography estimation methods have displayed greater performance in single-source image, but it is difficult to extract accurate common features in infrared and visible images, resulting in poor performance. This paper proposed an unsupervised homography estimation module for infrared and visible images. Firstly, the network extracts the mask and feature maps of visible and infrared images using a mask generator and a feature extractor that introduces the RDB module, respectively. The feature extraction ability of the network can be enhanced by RDB's utilization of the hierarchical features present in the various convolutional layers. Then, the feature maps and masks are multiplied to get the weighted feature maps. Finally, the feature maps with assigned weights are cascaded by channel and fed into the homography estimator Res-CBAM for acquiring the homography matrix. The Res-CBAM module utilizes ResNet-34 as backbone and highlights the features that are important for homography estimation, by inserting the CBAM module after each stage (except the last one). The experimental results show that the ACE of the proposed method reduces significantly from 5.25 to 5.12.