A column-based self-organizing hybrid storage model for data warehouse
Xiangwu Ding, Mei Wang, Jiajin Le · 2010
Storage structures of data are critically important for performance and scalability. In read-mostly analytic applications, column-store exhibits significant advantage over row store. In column-store, one of the most cost-time operations is on-the-fly tuple reconstruction for multi-attribute queries. In this paper we propose a column-based self-organizing hybrid storage model suiting data warehouse workload, also define query relativity between attributes in a relational table. In our proposed model, a given relational table is stored by column first, strongly-relative multiple columns within a relational table are solidified in store device according to users' conventions then. During the system's running, it dynamically tunes solidified multi-columns to adapt physical storage of data to usage convention, i.e. strongly relative multi-columns can be solidified together as in good time as possible so that achieving fundamental performance improvement. The proposed method also automatically removes existing solidified multi-columns being no longer strongly-relative, to avoid excessive expansion of stored data. Finally, we analyze the advantage of our model over alternative storage structures and apply the proposed model to our column-oriented data warehouse management system being developed.