A Cost-Efficient Metadata Scheme for High-Performance Deduplication Systems
Yuxuan Mo, Yu Hua, Pengfei Li, Qin Cao, Xue Liu · 2021
Data deduplication has been widely used in backup systems to eliminate redundant data, which speeds up the backup process and reduces the storage overhead. Deduplication packs multiple chunks into a large, fixed-size container as a storage unit to maintain the locality and achieve efficient compression. We observe that the traditional containers have low filling ratios due to a large amount of metadata generated by small files. Unfilled containers require more space to store a backup, which decreases the storage efficiency and reduces restore performance. In order to address this problem, we propose a Metadata region Adaptive Container Structure, called MACS. MACS maintains a tag to record the length of metadata region in the container. The boundary between meta-data region and data region is dynamically decided to ensure the maximum space efficiency of the containers. Moreover, we propose a container metadata length-based indexing and cache replacement strategy to allow MACS to be practical in data backup systems. We demonstrate the advantages of MACS with three real world backup datasets. MACS achieves over 95% average container filling ratio, which is significantly higher than existing designs. MACS further achieves better restore performance than the traditional container structure. When combined with existing rewriting method, MACS achieves an efficient trade-off between deduplication ratio and restore performance.