Dimensionally Unified Metric Model for Multisource and Multiview Scene Matching
Bo Cun Sun, Ganchao Liu, Yuan Yuan · IEEE Transactions on Geoscience and Remote Sensing · 2024
The core challenge of multi-view scene matching is to effectively extract features from multi-view images, which is a key factor to achieve robust scene matching. This paper delves into methodologies for enhancing the multi-view robustness of scene matching, with the following key contributions: 1) This paper propose the metric feature consistency principle which emphasize the necessity of implementing accurate correspondence between drone and satellite images. To verify the principle, consistent feature enhancement for channel, spatial, and hybrid dimensions is explored. As a counterexample, the shuffle operation is used to break the consistency of dimension semantic information. Experiments show that the mAP is reduced by about 20% after breaking the consistency relationship of metric features. 2) Built upon the principle, this paper introduces a scene matching framework named DUMM (Dimensionally Unified Metric Model). Utilizing the multi-dimensional feature enhancement model, it effectively enhances the correspondence of the dual-branch features within the siamese network. 3) This paper is the first to introduce the feature dimension, frequency domain feature. By the upper and lower bounds of the cosine transform function, the negative effects of cross-view variations can be effectively mitigated, thus enhancing the overall robustness. The introduction of frequency features improves the mAP by 2.54% compared with the method of improving the dimensional consistency ofH×W×C, verifying the positive impact of the fusion of frequency domain features on enhancing the robustness of scene matching. Nevertheless, it remains imperative to adhere to the principle of feature dimension consistency across all additional frequency dimensions.