Ensemble-Based System Benchmarking for HPC
Anna Fuchs, Jannek Squar, Michael Kühn · 2024
In HPC supercomputers, the CPU, memory, network and storage play a critical role in application performance. Established benchmarks measure the theoretical peak performance of these components, but storage benchmarks often focus solely on I/O and lack realism. To address this, we present numio, a benchmark that simulates and evaluates overlapping compute, communication and I/O phases. Furthermore, we present a novel ensemble-driven system benchmarking strategy. This approach involves running multiple benchmarks in parallel to analyse their interactions and assess the system’s ability to handle the workload. Using a real-world example, we demonstrate how this approach reveals performance issues in complex HPC systems that remain hidden when using traditional methods using isolated benchmarks on empty systems.