A scheme of structured data compression and query on Hadoop platform
Xiangwu Ding, Bo Tian, Yefeng Li · 2015
We proposed a scheme of data compression and query technology to improve the performance of processing structured data on Hadoop platform. Firstly, we designed a data page structure for row-column hybrid storage based on HDFS. Then we proposed and implemented an adaptive lightweight data compression strategy based on MapReduce to compress and store data as the proposed storage structure. Finally, we provided a query strategy which directly execute on the compressed data of the given storage structure. The experiments conducted on the large-scale datasets demonstrated the effectiveness of the proposed strategy on reducing the amount of storage and improving query performance for structured data.