Exploiting locality for query processing and compression in scientific databases
Anand Kumar, Yi-Cheng Tu · 2010
Improvements in the efficiency of scientific simulations have lead to requirements of large databases. The data captured in such simulations is of large scale and poses challenges in storage, transfer and query processing. However, the data are collected every fraction of a second, storing some redundant information. Thus, the temporal and spatial locality of the data gives us an opportunity to store and transfer over networks efficiently. The data locality also helps in efficiently processing complex analytical queries that are popular in scientific databases. Many scientific data analysis queries involve more than one object/body of interest. Processing such queries pose super-linear computational complexity. In this paper, we propose preliminary solutions to some of these problems along with initial results. Mainly, we try to exploit the spatial and temporal proximity of the data to achieve high levels of compression for efficient storage and analytical query processing.