From Independence to Cooperation: A Novel Keypoint Extraction Framework for Autonomous Systems
Shitao Chen, Haolin Zhang, Yuchen Yan, Nanning Zheng · 2023
Despite the numerous keypoint extraction methods that have been developed over the past decades, the practice of extracting keypoints for each image without knowledge of others remains unchanged. This independent treatment makes it difficult to pursue higher keypoint repeatability among image pairs. To address this issue, we propose a novel cooperative framework for keypoint extraction. Our framework includes a lightweight spatial-temporal attention module, called CoMod (cooperative module), that facilitates the interaction between image representations. By integrating CoMod into a Siamese encoder-decoder network, we introduce a novel keypoint detector CoKE (Cooperative Keypoint Extraction). Additionally, we design a semi-supervised decoupled learning mechanism to enable the effectiveness of CoKE. Our experiments on the HPatches benchmark demonstrate that our method achieves state-of-the-art keypoint repeatability performance and improves downstream robot vision tasks, which are significant to autonomous driving and intelligent transportation systems.