A Note on Performance Profiles for Benchmarking Software
Nicholas I. M. Gould, J. A. Scott · ACM Transactions on Mathematical Software · 2016
In recent years, performance profiles have become a popular and widely used tool for benchmarking and evaluating the performance of several solvers when run on a large test set. Here we use data from a real application as well as a simple artificial example to illustrate that caution should be exercised when trying to interpret performance profiles to assess the relative performance of the solvers.