Adaptive Stopping Rule for Performance Measurements
Viyom Mittal, Pedro Bruel, Dejan S. Milojicic, Eitan Frachtenberg · 2023
Performance variability in complex computer systems is a major challenge for accurate benchmarking and performance characterization, especially for tightly-coupled large-scale high-performance computing systems. Point summaries of performance may be both uninformative, if they do not capture the full richness of its behavior, and inaccurate, if they are derived from an inadequate sample set of measurements. Determining the correct sample set—and in particular, its size—requires balancing tradeoffs of computation, methodology, and statistical power.