Evolution and evaluation of SPEC benchmarks
Jozo J. Dujmovic, Ivo Dujmovic · ACM SIGMETRICS Performance Evaluation Review · 1998
We present a method for quantitative evaluation of SPEC benchmarks. The method is used for the analysis of three generations of SPEC component-level benchmarks: SPEC89, SPEC92, and SPEC95. Our approach is suitable for studying (1) the redundancy between individual benchmark programs, (2) the size, completeness, density and granularity of benchmark suites, (3) the distribution of benchmark programs in a program space, and (4) benchmark suite design and evolution strategies. The presented method can be used for designing a universal benchmark suite as the next generation of SPEC benchmarks.