When Should I Run My Application Benchmark?
Sören Henning, Adriano Vogel, Esteban Pérez-Wohlfeil, Otmar Ertl, Rick Rabiser · 2025
Performance benchmarking is a common practice in software engineering, particularly when building large-scale, distributed, and data-intensive systems. While cloud environments offer several advantages for running benchmarks, it is often reported that benchmark results can vary significantly between repetitions—making it difficult to draw reliable conclusions about real-world performance.