Understanding DAOS Storage Performance Scalability
Michael Hennecke · 2023
High performance scale-out storage systems are a critical component of modern HPC and AI clusters. However, characterizing their performance remains challenging: Different client I/O patterns have very different performance scaling behavior, and bottlenecks in the HPC storage software may also limit performance scaling. The Distributed Asynchronous Object Storage (DAOS) is an open source scale-out storage system that is designed from the ground up to support Storage Class Memory (SCM) and NVMe storage in user space. DAOS has demonstrated leadership performance on IO500. But while IO500 has defined an extensive “standard” set of I/O patterns, a single IO500 run does not provide much insight into how a storage solution scales. This paper investigates the performance scaling behavior of the DAOS storage stack for typical IOR and mdtest workloads, as a function of the storage server hardware and the client-side parallelism. It also analyses the impact of the HPC networking stack (libfabric/verbs and UCX) on the achievable performance.