UHUM: An Efficient Hybrid Update Mechanism in Distributed Storage Systems with Erasure Coding
Qian Luo, Yun Wang · 2019
With the explosive growth of data, many distributed storage systems have incorporated erasure coding technology to replace replication to reduce storage costs. Most state-of-art distributed storage systems with erasure coding are based on log appending and do not support in-place updates. In this paper, we focus on modeling two update mechanisms of RAID, reconstruct writes and read-modify writes, in distributed storage systems. To the best of our knowledge, we first p ropose a novel update mechanism called update-size-aware hybrid update mechanism (UHUM), which dynamically chooses a better update mechanism based on update size and mitigates the network transfer. An erasure-coded distributed system called EcDFS is further built with strong consistency among data chunks and parity chunks. Testbed experiments are conducted on different update mechanisms in WAN and LAN. The results show that EcDFS efficiently reduces update latency, especially in WAN, up to about 28.1% and 24.2% respectively lower than reconstruction writes and read-modify writes.