G2PTL: A Geography-Graph Pre-trained Model
Lixia Wu, Jianlin Liu, Junhong Lou, Minhui Deng, Jianbin Zheng, Haomin Wen, Chao Song, Shu He · 2024
As an important data resource containing spatial information, addresses record the geospatial information corresponding to social production activities and human behavioral activities. How to effectively encode addresses has always been a core challenge in the field of Geographic Information Systems (GIS). Pre-trained Models (PTMs) designed for Natural Language Process (NLP) have emerged as the dominant tools for encoding semantic information in text. Though promising, those NLP-based PTMs fall short of encoding geographic knowledge in addresses, which limits their application potential in geospatial tasks. To tackle the above problem, this study proposes a Geography-Graph Pre-trained model (G2PTL) that combines graph learning and text pre-training, aiming to make up for the shortcomings of traditional PTM in the geography field. Specifically, we first utilize real-world delivery data to build a large-scale heterogeneous graph of addresses, which contains abundant geographic knowledge and spatial topology information. Then, G2PTL is pre-trained with subgraphs sampled from the heterogeneous graph. Through experimental evaluation on multiple downstream tasks of GIS, including geocoding, geographic entity prediction, and geographic entity recognition, G2PTL demonstrated significant performance improvements. G2PTL has been successfully deployed in production-level GIS, such as Cainiao's logistics system, effectively improving the execution efficiency and accuracy of address-related tasks. This research not only provides a new technical path for the encoding and processing of geographical information, but also opens up a new perspective for the study of pre-training models in the geographical field. The code resources of the G2PTL model have been opened for research and application developers to access and use at https://huggingface.co/Cainiao-AI/G2PTL.