Multi-level Byte Index Chunking Approach for File Synchronization
Ider Lkhagvasuren, Jung Min So, Jeong Gun Lee, Young Woong Ko · 2012
In this paper, we propose a multi-level byte index chunking algorithm that efficiently compresses data file. The proposed system produces two types of double Index table for a file. Each chunk sizes are 4MB and 32KB, respectively. At the first level, system client detects large sized identical data blocks using 4MB chunk sized index-table by using byte Index chunking approach in fast time. At the second level, we perform byte-index chunking using 32KB index-table on entire non-duplicated data produced through first level file similarity detection. This gives us opportunity to more accuracy rated data deduplication and doesn't consume so much time because deduplication work restricted by only non-duplicated area.