Palantir: Hierarchical Similarity Detection for Post-Deduplication Delta Compression
Hongming Huang, Peng Wang, Qiang Su, Hong Li Xu, Chun Jason Xue, André Brinkmann · 2024
Deduplication compresses backup data by identifying and removing duplicate blocks. However, deduplication cannot detect when two blocks are very similar, which opens up opportunities for further data reduction using delta compression. Most existing works find similar blocks by characterizing each block by a set of features and matching similar blocks using coarse-grained super-features. If two blocks share a super-feature, delta compression only needs to store their delta for the new block.