LLAMP: Assessing Network Latency Tolerance of HPC Applications with Linear Programming
Siyuan Shen, Langwen Huang, Marcin Chrapek, Timo Schneider, Jai Dayal, Manisha Gajbe, Robert W. Wisniewski, Torsten Hoefler · 2024
The shift towards high-bandwidth networks driven by AI workloads in data centers and HPC clusters has unintentionally aggravated network latency, adversely affecting the performance of communication-intensive HPC applications. As large-scale applications often exhibit significant differences in their network latency tolerance, it is crucial to determine the extent of network latency an application can withstand without significant performance degradation. Current approaches often rely on specialized hardware or simulators, which can be inflexible and time-consuming. We introduce LLAMP, a novel toolchain that offers an efficient analytical approach to evaluating HPC applications’ network latency tolerance using the LogGPS model and linear programming. Through our validation on a variety of MPI applications such as LULESH and MILC, we demonstrate our tool’s high accuracy, with relative prediction errors below 2%. Additionally, we include a case study of the ICON weather and climate model to illustrate LLAMP’s broad applicability in evaluating collective algorithms and network topologies.