Scalable and efficient spatial data management on multi-core CPU and GPU clusters: A preliminary implementation based on Impala
Simin You, Jianting Zhang, Le Gruenwald · 2015
Fast increasing volumes of spatial data has made it imperative to develop both scalable and efficient spatial data management techniques by leveraging modern parallel hardware and distributed systems. By integrating a leading open source Big Data system called Impala and our previous work on data parallel designs for spatial indexing and query processing, we have developed ISP-MC+ and ISP-GPU for large-scale spatial data management on computer clusters equipped with multi-core CPUs and Graphics Processing Units (GPUs), respectively. Both ISP-MC+ and ISP-GPU have shown high efficiency and good scalability on a 10-node Amazon EC2 cluster equipped with multi-core CPUs and GPUs. Comparison with a baseline implementation using traditional techniques on a single CPU core have demonstrated orders of magnitude of speedups on a real world dataset with hundreds of millions of point locations.