Lightweight Dynamic Redundancy Control for Server-Based Storage
Takayuki Fukatani, Hieu Le, Haruo Yokota · 2019
The recent performance improvements in commodity hardware have made commodity server-based storage a practical alternative to dedicated-storage appliances. Because of the low reliability of commodity servers, data redundancy across multiple servers is required for high availability of a server-based storage system. However, the extra storage capacity required to enable this redundancy increases the system cost significantly. Although erasure coding (EC) is a promising approach to reducing the amount of redundant data, it is only available in systems using distributed storage. There remains the need to reduce the performance overhead of using distributed storage, which involves much network traffic and metadata processing. In this paper, we propose a lightweight redundancy-control method called "dynamic redundancy control" for server-based storage systems. Our method adds additional file metadata for EC to the local filesystem, enabling the system to take advantage of EC with reduced network traffic and metadata processing. In addition, our method dynamically controls the data redundancy in user data between replication and EC to improve the capacity efficiency while mitigating performance degradation. Our experiments show that our method achieves up to 76% less processing time for a metadata-intensive workload, up to 150% higher read performance, and up to 147% better online-transaction-processing performance than CephFS, a widely used alternative system. In addition, our method successfully improves capacity efficiency while mitigating performance degradation.