Neptune: Efficient remote communication services for cloud backups
Yu Hua, Xue Liu, Dan Feng · 2014
In order to efficiently achieve fault tolerance in cloud computing, large-scale data centers generally leverage remote backups to improve system reliability. Due to longdistance and expensive network transmission, the backups incur heavy communication overheads and potential errors. To address this important problem, we propose an efficient remote communication service, called Neptune. Neptune efficiently transmits massive data between long-distance data centers via a cost-effective filtration scheme. The filtration in Neptune is interpreted as eliminating redundancy and compressing similarity of files, which are generally studied independently in existing work. In order to bridge the gap between them, Neptune leverages chunk-level deduplication to eliminates duplicate files, and approximate delta compression to compresses similar files. Moreover, in order to reduce the complexity and overheads, Neptune uses a locality-aware hashing to group similar files and proposes shortcut delta chains for fast remote recovery. We have really implemented Neptune. We examine the Neptune performance by using real-world traces of LANL, HP, MSN and Google. Compared with state-of-the-art work, experimental results demonstrate the efficiency and efficacy of Neptune.