Bitmap indexes for large scientific data sets: A case study
Rishi Rakesh Sinha, Soumyadeb Mitra, Marianne Winslett · 2006
The data used by today’s scientific applications are often very high in dimensionality and staggering in size. These characteristics necessitate the use of a good multidimensional indexing strategy to provide ef-ficient access to the data. Researchers have previously proposed the use of bitmap indexes for high-dimension scientific data as a way of overcoming the drawbacks of traditional multidimensional indexes such as R-trees and KD-trees, which are bulky and whose performance does not scale well as the number of dimensions in-creases. However, the techniques proposed in previous work on bitmap indexes are not sufficient to address all problems that arise in practice. In experiments with real datasets, we experienced problems with index size and query performance. To overcome these shortcom-ings, we propose the use of adaptive, multilevel, multi-resolution bitmap indexes, and evaluate their perfor-mance in two scientific domains. Our preliminary ex-periments with a parallel query processor and index cre-ator also show that it is very easy to parallelize a bitmap index. 1