Rethinking Software Defined Cloud Storage for Disaggregation
Yi Zou, Arun Raghunath, Anjaneya Chagam, Sujoy Sen, Tushar Gohad · 2019
Software Defined Storage (SDS) architectures abstract away underlying physical hardware and provide durability, availability and reliability guarantees by transparently creating replicas, erasure coding, and performing periodic background data checks. However, current SDS architectures are at odds with the evolving trend of Data Centers towards physically separating server resources like storage, into distinct sleds connected via high speed interconnects such as NVMe over Fabric (NVMe-oF), for finer grained control and flexibility. The assumption of locally attached storage and a monolithic server design exacerbate the overheads of the internal operations performed within the SDS framework in a disaggregated Data Center. We propose redefining the boundaries of separation within SDS architectures to address disaggregation overheads. Specifically, we decouple control and data plane operations and transfer block ownership to execute on remote storage targets. This redistribution of responsibilities provides a 40% reduction in bandwidth overheads and a 2X end-to-end latency reduction associated with internal SDS data replication traffic. Using Ceph as an example, we provide detailed analysis on the potential reduction in bandwidth and latency on the proposed architecture. We demonstrate decoupling the Ceph BlueStore functionality and integrating with NVMe-oF target layer. We further present preliminary results collected on a testing PoC Ceph cluster currently being developed.