Optimising column family for OLAP queries in HBase
Fangzhou Yang, Dragan Milosevic, Jian Cao · International Journal of Big Data Intelligence · 2016
Apache HBase is a column-oriented NoSQL key-value store built on top of the Hadoop distributed file-system. Logically, columns in HBase are grouped into column families. Physically, all columns in one column family are stored in the same set of files. Therefore the division of column families is closely related to the response time for a specific row query. In this paper, a new evolutionary algorithm is designed and applied to group columns and find the optimum column family schema for the given user queries. The reading performance of the optimised column family schema is evaluated on a real dataset provided by ZANOX. It is shown that by using the found optimised column family schema, the reading performance while executing OLAP queries is improved with a statistical significance. Queries from a test set show that the average response time is reduced by up to 72% compared to reference column family schemas.