Towards code metrics for benchmarking timing analysis
Peter Wägemann, Tobias Distler, Phillip Raffeck, Wolfgang Schröder‐Preikschat · 2016
Comprehensive evaluations of the effectiveness of worst-case execution time (WCET) analyzers require a selection of benchmarks that pose a challenge to these tools. In this paper, we identify pitfalls that are associated with selecting such benchmarks based on complexity metrics (e.g., the number of loops contained in a program), which in part are caused by the fact that complexity measures are not necessarily stable in the face of compiler optimizations. To address these problems, we are developing a tool that automatically assesses the resilience of a benchmark against compiler optimizations by tracking complexity measures across different optimization levels. In combination with information on the data dependency of control flows, which is also provided by our tool, this allows users to find and discard benchmarks that appear challenging for WCET analyzers at the source-code level, but in fact are trivial at the machine-code level where the actual analysis is performed.