Adaptive Erasure Coded Data Maintenance for Consensus in Distributed Networks
Yulei Jia, Guangping Xu, Chi Wan Sung, Salwa Mostafa · 2021
Distributed data services usually rely on consensus protocols, such as Paxos and Raft, to provide fault-tolerance and data consistency across distributed data centers and even edge networks. In consensus protocols, erasure coded replication has appealing storage and network cost savings compared with full copy replication, which help achieve low latency, high fault-tolerance and high throughput. However, the liveness level will inevitably decrease when erasure codes are naively applied in consensus protocols. To keep the original liveness level, an existing protocol, called CRaft, switches from erasure coded replication to full copy replication when the number of failures exceeds a certain threshold. Such a solution, however, degrades system performance sharply. To tackle this problem, this work proposes a novel protocol called HRaft to enable graceful degradation on storage and network efficiency when failures happen. Without using full copy replication, it replenishes some coded blocks in healthy servers to reduce storage and network costs and to keep data consistency. The performance of the proposed protocol will be evaluated by deploving it into practical networks.