Benchmarks as measures of performance: the Los Alamos approach
Harvey J. Wasserman · Transactions of the American Nuclear Society · 1987
The computer research and applications group of the Los Alamos National Laboratory (LANL) has been involved in the evaluation of computing performance for many years. The benchmarking effort consists of annual tests on computing systems currently in use at LANL as well as evaluation of new computers that are not in LANL's central computing facility. Attention has been focused primarily on supercomputers, although, for various reasons, less powerful systems are sometimes considered. Supercomputer benchmarking is a nontrivial task. The benchmark methodology employed at LANL involves several ground rules: (1) currently, LANL only performs uniprocessor tests, even on multiprocessor machines; (2) in the past LANL has not measured input/output (I/O) performance; however, I/O benchmarks are being developed; (3) these benchmark codes consist entirely of Fortran. The codes are computationally intensive, with generally little dependence on operating systems; (4) a benchmark always consists of determining performance at three levels of code tuning: (a) only changes necessary to get the codes running; (b) codes optimized for a new machine with a reasonable investment of programming time; (c) optimal performance.