ER-KV: High Performance Hybrid Fault-Tolerant Key-Value Store
Yingjie Geng, Jinfei Luo, Gang Wang, Xiaoguang Liu · 2021
In-memory Key-Value store (KV-store) has been widely used to meet increasing performance requirements. Such system stores user data in main memory and when a node fails, all data will be lost due to the volatile nature of DRAM. Therefore, it usually takes fault-tolerance mechanisms such as primary-backup replication (PBR) and erasure coding to ensure reliability and availability. PBR achieves high availability and scalability at the cost of high data redundancy. Erasure coding has higher utilization of storage but introduces additional computing and network overhead. This paper presents a high-performance fault-tolerant distributed KV-store, the Erasure-coded Replication KV (ER-KV), which takes advantage of both erasure coding and PBR by applying a two-level fault-tolerant mechanism. To achieve both fast recovery speed and low storage overhead, ER-KV gives priority to PBR for fault tolerance but will turn to erasure coding for data recovery when backup nodes are all offline. When generating parity data, ER-KV leverages erasure coding for large value while applying PBR to small-sized key to avoid frequent encoding and decoding. ER-KV also introduces pipelining into erasure coding to improve system performance, and simultaneously, ER-KV still maintains data consistency. Besides, ER-KV applies persistent memory (PM) to achieve the persistence of redundant data and improve recovery speed. We have applied ER-KV to Redis and the experimental results show that it saves 33% of storage compared with PBR. As for erasure coding, ER-KV has higher recovery efficiency and system availability. And it also brings 15X-80X speedup in data reconstruction when applying PM to backup nodes.