Factors Causing Unexpected Variations in Ada Benchmarks.
Neal Altman · 1987
Benchmarks are often used to describe the performance of computer systems. This report considers factors that may cause Ada benchmarks to produce inaccurate results. Included are examples from the ongoing benchmarking efforts of the Ada Embedded Systems Testbed (AEST) Project using bare target computers with several Ada compilers. 1. Introduction One of the goals of the Ada Embedded Systems Testbed (AEST) Project is to assess the readiness of the Ada programming language and Ada tools for developing embedded systems. The benchmarking and instrumentation subgroup within the AEST Project is running various suites of Ada benchmarks to obtain data on the real-time performance of Ada on a number of different target systems. The purpose of this report is to categorize the factors which cause anomalous results to be produced by the benchmarks. Some of these factors have been observed, while others are more speculative in nature. All these factors should be understood if accurate, comparable,...