GIS spatial data partitioning method for distributed data processing
Yan Jie Zhou, Qing Zhu, Yeting Zhang · Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE · 2007
Spatial data partitioning strategy plays an important role in GIS spatial data distributed storage and processing, its key problem is how to partition spatial data to distributed nodes in network environment. Existing main spatial data partitioning methods doesn't consider spatial locality and unstructured variable length characteristics of spatial data, these methods simply partition spatial data based on one or more attributes value that could result in storage capacity imbalance between distributed processing nodes. Aiming at these, we point out the two basic principles that spatial data partitioning should meet to in this paper. We propose a new spatial data partitioning method based on hierarchical decomposition method of low order Hilbert space-filling curve, which could avoid excessively intensive space partitioning by hierarchically decomposing subspaces. The proposed method uses Hilbert curve to impose a linear ordering on the multidimensional spatial objects, and partition the spatial objects according to this ordering. Experimental results show the proposed spatial data partitioning method not only achieves better storage load balance between distributed nodes, but also keeps well spatial locality of data objects after partitioning.