Practical Amplification of Condition/Decision Test Coverage by Combinatorial Testing
Artur Andrzejak, Thomas Bach · 2018
Test suites in complex software projects might grow over time to considerable sizes, incurring high maintenance effort and prolonged execution times. Maintaining their quality and efficiency require pruning of redundancies while increasing, or at least retaining their coverage levels. We propose a lightweight method to tackle these problems with focus on condition/decision coverage (C/D coverage). First, we describe a method to reduce the size of unit tests suites while preserving the degree of their C/D coverage. We then introduce an approach which combines combinatorial testing and input space modeling to further increase the degree of the C/D coverage. Our semi-automated method works even in absence of models or documentation, and it produces a low number of new test cases requiring queries to a test oracle. We also do not use symbolic execution techniques due to their complexity and limited tool availability for some languages. These properties make our approach practically applicable in industrial projects, and simpler to implement. We evaluate our approach on selected examples from SAP HANA, a very large industrial application in C++. We demonstrate that it is possible to generate from integration tests new suites of unit tests with high C/D-coverage but with only few test cases. At the same time, the human effort of creating such suites is moderate.