Chunking implementation of extendible array to handle address space overflow for large multidimensional data sets
K. M. Azharul Hasan, Mehnuma Tabassum Omar, Sheikh Mohammad Masudul Ahsan, Nazmin Nahar · 2013 International Conference on Electrical Information and Communication Technology (EICT) · 2014
Multidimensional array is commonly used in data warehouse or in many scientific computation of OLAP for dealing with application dataset. Different models of multidimensional array have been introduced to do this job like Traditional Multidimensional Array, Extendible array, Extended Karnaugh Map Representation etc. But Application that requires operation on large amount of data cannot be handled by these models. One common approach is to store data in chunked multidimensional arrays. Indices into an array are obtained by a normalization process that maps attribute values to integers. We have introduced an approach of storing huge amount of data contiguously in disk by applying the chunking approach to an extendible multidimensional array. This implemented approach is effective in managing memory than any other existing approach such as traditional multidimensional array. In our system data are stored in memory according to some predefined amount of memory space that will help the memory to store large amount of data through small unit of chunk where the size of chunk will never overflow the address space limit of a computer.