High Performance Multidimensional Analysis and Data Mining
Sanjay Goil, Alok Choudhary · 1998
Summary information from data in large databases is used to auswer queries in On-Line Analytical Processing (OLAP) systems and to build decision support systems over them. The Data Cube is used to calculate and store summary in-formation on a variety of dimensions, which is computed only partially if the number of dimensions is large. Queries posed on such systems are quite complex and require dif-ferent views of data. These may either be auswered from a materialized cube in the data cube or calculated on the fly. Further, data mining for associations can be performed on the data cube. Analytical models need to capture the multi-dimensionality of the underlying data, a task for which mul-tidimensional databases are well suited. Multidimensional databases tore data in multidimensional structure on which analytical operations are performed. A challenge for these systems is how to handle large data sets in a large number of dimensions. This paper presents q parallel OLAP h&structure for multidimensional databases integrated with association rule mining. Scheduling optimisations for parallel computation of wmplete data cubes are presented. We propose left aud right schedules for partial data cubes for m-way mining of association rules. Our implementation on the IBM SP-2, a shared-nothing parallel machine, cau handle large data sets and a large number of dimensions by using disk I/O in our algorithms. 1