A new benchmark test to estimate optimization quality of compilers
Melvin Klerer, Hong Liu · ACM SIGPLAN Notices · 1988
We have just completed an empirical study of the frequency of certain programming structures.These are eleven programming structures that are most commonly discussed in compiler textbooks as susceptible to optimization.The results of this study will be published elsewhere.Our sample space dealt with programs in Fortran, Pascal, C, and Basic, written by Computer Science and non-Computer Science undergraduate and graduate students as well as professors and Ph.D research candidates from Computer Science and non-Computer Science departments.Programs were classified as primarily numerical, control, or data manipulation.The length of these programs ranged from short exercises to programs of several thousand source-code statements.Using the results of this study we have constructed a new benchmark test to estimate the optimization quality of compilers.This test consists of two programs, "NON-OP" and "OP".The program NON-OP contains eleven sets of program structures, each of which is iterated a number of times equal to the statistical weight observed for each structure in our empirical sample.The program OP is computationally equivalent to NON-OP, but is written in "perfect" programming style so that its execution time will not be affected by an optimizing compiler.