Diablo Test Suite: Code Coverage Analysis
Rishi K. Ganeriwala, USDOE National Nuclear Security Administration (NNSA) · 2019
An analysis of the Diablo source code covered by its Software Quality Assurance test suite was performed. Intel’s code coverage tool was used to determine the percentage of covered files, blocks, and functions. Excluding non-functional legacy files and single use developer-only files, the code coverage tool showed that 55% of blocks, the most applicable metric and closest to lines of code, are covered. For a large software code, 70-80% coverage is generally considered a reasonable target.A significant percentage of the uncovered code can be categorized as features that are not practical to test, duplicate/unused legacy code, and niche features only used by select individuals. Additionally, large segments of code dedicated to debugging and error handling are untested, as the test problems do not error out. The remaining untested code can be considered general purpose for use by the analyst/user community. These untested general purpose features are identified in the document and should be prioritized when new testing is added. Suggestions for additional test problems that would have the biggest impact on increasing the code coverage metric are presented. Finally, it is important to note that the single biggest way to increase the overall coverage metric is to delete old and unused code. While admittedly tedious to perform, future code development would also benefit markedly from this. Deleting unused code would likely push the coverage metric north of 60%, which is not too far off the 70-80% target. Overall, the current test suite covers the majority of commonly used features in Diablo, but there remains significant room for improvement. These areas for improvement have been identified.