Improving main memory utilization for array-based datacube computation
Seigo Muto, Masaru Kitsuregawa · 1998
Computing datacubes requires multidimensional aggregations for all possible combinations of each dimension.In thii paper, we present a method to improve main memory utilization efficiency for an array-based algorithm for datacube computation in a MOLAP context.The problem with the array-based algorithm is in its sparsity, where a large pre portion of array cells are empty.,The algorithm proposed in [ZDN97] reduces this space inefficiency by compressing arrays on disk.We improve on this algorithm by performing compression of arrays in main memory as well as on disk using a hashing method, which allocates main memory according to the number of non-empty array cells.We further improve the algorithm using a dynamic main memory allocation strategy.The algorithm by [ZDN97] computes the multiple aggregate views simultaneously, which consumes a lot of main memory space.We propose a main memory allocation method that minimizes the main memory requirement by dynamically allocating main memory only to necessary aggregate views at run time.These savings in main memory resources result in the reduction of disk I/O cost.We evaluate the performance of the proposed method by disk I/O analysis and demonstrate that the improved MOLAP algorithm compares well with a ROLAP algorithm.Permission 10 make digital or hard copies of all or part of this work for personal ur classroom use is granted without fee pro$ided that copies arc not made or distributed for protit or comnwciai advantage and that cop&s bear this notice and the full citation on Ihc firS1 pa!& 'f0 COPY othemzise.to republish.to post on sewers or to redistribute to lists.requires prior specific prrmission an&W a fW.