DAOS as HPC Storage: Exploring Interfaces
Adrian Jackson, Nicolau Manubens · 2023
Reading and writing data to and from data storage has long been a bottleneck for high performance computing (HPC). The hardware used for data storage exhibits performance that is generally at least an order of magnitude lower than that of volatile memory or processing hardware. Applications often limit the amount of I/O undertaken on HPC systems to reduce this cost, but recently there has been a rise in application categories where ingestion or production of large amounts of data is common, machine learning being an obvious example. As HPC systems increase in size, reaching Exascale levels and beyond, and a subset of applications require ever larger amounts of I/O bandwidth or metadata performance, there is a significant challenge to improve the performance of the data storage technologies employed for I/O operations.