Efficiently Retrieving Differences Between Remote Sets using Counting Bloom Filter
Xiaomei Tian · International Journal of Performability Engineering · 2019
Retrieving differences between remote sets is widely used in set reconciliation and data deduplication.Set reconciliation and data deduplication between two nodes are widely used in various network applications.The basic idea of the difference retrieving problem is that each member of a node pair has an object set and seeks to find all differences between the two remote sets.There are many methods for retrieving difference sets, such as the standard Bloom filter (SBF), counting Bloom filter (CBF), and invertible Bloom filter (IBF).In these methods, based on the standard Bloom filter or its variants, each node represents its objects using a standard Bloom filter or other Bloom filter, which is then exchanged.A receiving node retrieves different objects between the two sets according to the received SBF, CBF, or IBF.We propose a new algorithm for retrieving differences that finds differences between remote sets using counting Bloom filters' deletion operation.The theoretical analyses and experimental results show that the differences can be retrieved efficiently.Only a very small number of differences are missing in the retrieving process, and this false negative rate can be decreased to 0% by adjusting the counting Bloom filter's parameters.