Achieving Tunable Erasure Coding with Cluster-Aware Redundancy Transitioning
F.-Z. Zhang, Fulin Nan, Binbin Xu, Zhirong Shen, Jiebin Zhai, Dmitrii Kalplun, Jiwu Shu · ACM Transactions on Architecture and Code Optimization · 2024
Erasure coding has been demonstrated as a storage-efficient means against failures, yet its tunability remains a challenging issue in data centers, which is prone to induce substantial cross-cluster traffic. In this article, we presentClusterRT, a cluster-aware redundancy transitioning approach that can dynamically tailor the redundancy degree of erasure coding in data centers.ClusterRTformulates the data relocation as the maximum flow problem to reduce cross-cluster data transfers. It then designs a parity-coordinated update algorithm, which gathers the parity chunks within the same cluster and leverages encoding dependency to further decrease the cross-cluster update traffic.ClusterRTfinally rotates the parity chunks to balance the cross-cluster transitioning traffic across the data center. Large-scale simulation and Alibaba Cloud ECS experiments show thatClusterRTreduces 94.0% to 96.2% of transitioning traffic and reduces 70.4% to 88.4% of transitioning time.