Fine-grained building function recognition with street-view images and GIS map data via geometry-aware semi-supervised learning
Weijia Li, Jinhua Yu, Dairong Chen, Yi Lin, Runmin Dong, Xiang Zhang, Conghui He, Haohuan Fu · International Journal of Applied Earth Observation and Geoinformation · 2025
The diversity of building functions is vital for urban planning and optimizing infrastructure and services. Street-view images offer rich exterior details, aiding in function recognition. However, street-view building function annotations are limited and challenging to obtain. In this work, we propose a geometry-aware semi-supervised method for fine-grained building function recognition, which effectively uses multi-source geoinformation data to achieve accurate function recognition in both single-city and cross-city scenarios. We restructured the semi-supervised method based on the Teacher–Student architecture into three stages, which involve pre-training for building facade recognition, building function annotation generation, and building function recognition. In the first stage, to enable semi-supervised training with limited annotations, we employ a semi-supervised object detection model, which trains on both labeled samples and a large amount of unlabeled data simultaneously, achieving building facade detection. In the second stage, to further optimize the pseudo-labels, we effectively utilize the geometric spatial relationships between GIS map data and panoramic street-view images, integrating the building function information with facade detection results. We ultimately achieve fine-grained building function recognition in both single-city and cross-city scenarios by combining the coarse annotations and labeled data in the final stage. We conduct extensive comparative experiments on four datasets, which include OmniCity, Madrid, Los Angeles , and Boston, to evaluate the performance of our method in both single-city (OmniCity & Madrid) and cross-city (OmniCity - Los Angeles & OmniCity - Boston) scenarios. The experimental results show that, compared to advanced recognition methods, our method improves mAP by at least 4.8% and 4.3% for OmniCity and Madrid, respectively, while also effectively handling class imbalance. Furthermore, our method performs well in the cross-categorization system experiments for Los Angeles and Boston, highlighting its strong potential for cross-city tasks. This study offers a new solution for large-scale and multi-city applications by efficiently utilizing multi-source geoinformation data, enhancing urban information acquisition efficiency, and assisting in rational resource allocation.