Sector-Based Compression and Compression Strategy Selection Method for Column Stores
Zhen Wang · Chinese Journal of Computers · 2010
Compression technology is an important research field in column-oriented management system.However,most previous compression techniques for column-oriented data use same algorithm for all columns,ignoring the local distribution of data,which greatly degrade the compression performance.This paper proposes a sector-based compress pattern,under such pattern further provides a novel learning-based compression strategy selection method for column stores.First,data column is divided into sectors in the method.The neighbor sector information and the statistic information of the column with the given sector respectively are extracted as two references.Then by learning the similarity between the reference and the given sector the recommended compression strategy can be obtained.Finally,the recommended compression strategy is improved by partly learning the given sector to guarantee the effectiveness of it.The experimental results on data warehouse benchmark data set SSB testify the effectiveness of the proposed method.