Accelerating Application Bulk Synchronous Writes in HPC Environments
Awais Ali Khan, Christopher J. Zimmer, Scott Atchley, Ross Miller, Sarp Oral, Feiyi Wang · 2024
High-bandwidth storage tiers are becoming more common for their capability to absorb high-rate, bursty I/Os. Notably, the designs of these fast storage tiers differ from system to system. The variation of these layers and non-uniform methods of access can pose challenges for applications seeking to run at multiple HPC facilities. Therefore, in this work, we present Spectral, a rapid-output abstraction library to accelerate application, bulk-synchronous writes on HPC systems. We design Spectral to enable applications to use high-bandwidth storage, such as node-local storage and distributed, write-caches (e.g., burst buffers) transparently without requiring modifications to the application or file system source code. The key idea is to allow applications to spend most of the time performing productive work and to not require any source code changes for maximum portability on different HPC architectures. Spectral internally re-routes write-only files through available, high-performance I/O resources before ultimately migrating them to the shared global parallel file system. For instance, on Summit, Spectral transparently places application outputs on node-local storage and then utilizes asynchronous migration to the center-wide GPFS file system. We evaluate Spectral on the Summit HPC system (1024 nodes) using the IOR benchmark and real scientific applications. Spectral shows linear performance scaling, improving application write performance by over an order of magnitude when compared to GPFS.