Pyro: a spatial-temporal big-data storage system
Li Shen, Shaohan Hu, Raghu Ganti, Mudhakar Srivatsa, Tarek Abdelzaher · 2015
With the rapid growth of mobile devices and applica-tions, geo-tagged data has become a major workload for big data storage systems. In order to achieve scalability, existing solutions build an additional index layer above general purpose distributed data stores. Fulfilling the se-mantic level need, this approach, however, leaves a lot to be desired for execution efficiency, especially when users query for moving objects within a high resolution geometric area, which we call geometry queries. Such geometry queries translate to a much larger set of range scans, forcing the backend to handle orders of mag-nitude more requests. Moreover, spatial-temporal ap-plications naturally create dynamic workload hotspots1, which pushes beyond the design scope of existing solu-tions. This paper presents Pyro, a spatial-temporal big-data storage system tailored for high resolution geometry queries and dynamic hotspots. Pyro understands geome-tries internally, which allows range scans of a geometry query to be aggregately optimized. Moreover, Pyro em-ploys a novel replica placement policy in the DFS layer that allows Pyro to split a region without losing data locality benefits. Our evaluations use NYC taxi trace data and an 80-server cluster. Results show that Pyro reduces the response time by 60X on 1km×1km rectan-gle geometries compared to the state-of-the-art solutions. Pyro further achieves 10X throughput improvement on 100m×100m rectangle geometries2. 1