Fingerprints for Compressed Columnar Data Search
Carmen Kwan · 2019
To enhance performance in main memory databases, compression techniques have been suggested to keep large volume of data in-memory, as opposed to loading data on demand from slower media storage. High compression ratio, however, comes with both memory and performance overhead for queries; packed data needs to be decompressed into vectors before applying optimized scan algorithms. In this work, we propose data summaries at column block level. Our preliminary experimental studies on TPC-H data confirm that under the same memory budget used for MinMax synopsis, our block headers can lower the false positive rates by up to 30% for compressed data scans and can reduce the overhead of employing advanced compression schemes.