An Optimized Method of Translating SQL to More Efficient Map-reduce Tasks
Jin Cao, Honglin Han, Mingming Zhao, Sijing Ye, Dehai Zhu, Lin Li · International Journal of Grid and Distributed Computing · 2015
Today it is common for map-reduce programs to be created from higher-level programming systems, which is often an implementation of SQL.In this paper, we propose an optimized method of translating SQL query to map-reduce tasks, which can be performed more efficiently by distributed computing systems.Our method introduces an optimization strategy which is generally applied in the procedure of merging query-plantree, an intermediate data structure derived from SQL abstract-syntax-tree, and this mainly helps us to achieve a considerable reduction on the number of generated mapreduce tasks and efficiency improvement of map-reduce implementation.Experimental evaluations performed by means of TPC-H tests show that our method significantly outperforms Hive in terms of reliability and efficiency.