Local Consensus Transformer for Correspondence Learning
Gang Wang, Yufei Chen · 2023
Correspondence learning is a vital component in multi-view geometry and computer vision. The heavy outliers make the matching problem very challenging. By revisiting the local consensus benefits of traditional feature matching, we introduce the local consensus to design a learnerable neural network to capture underlying correspondences, dubbed the Local Consensus Transformer, for wide-baseline stereo. Specifically, our network architecture consists of three operations. In order to construct the neighbor topology, a dynamic graph-based embedding layer is used first. These local topologies then guide the multihead self-attention layer to mine a greater amount of context through channel attention. After that, order-aware graph pooling is applied to extract global context from the embedded local consensus. Experimentally, the abalation study shows that pointnet-like learning models can benefit from local consensus. The proposed model achieves state-of-the-art performance on both the YFCC100M outdoor and SUN3D indoor challenging scenes with more than 90 percent outliers.