Ensuring Reliability Through Combinatorial Coverage Measures
Mohammad S. Raunak, D. Richard Kuhn, Raghu N. Kacker, Yu Lei · IEEE reliability magazine · 2024
Verification of complex software systems is an important, yet challenging task. Testing is the most common method for assuring that software meets its specifications and is defect-free. To claim that software is defect-free and thusreliable, one has to show that it produces the “correct” output or “behaves” according to specification without failing underallpossible parameter values and configurations. In the software verification world, this is known asexhaustive testing.For any software of reasonable size and complexity, exhaustive testing is completely infeasible. Thus, during the testing process, a small subset of parameter values and configurations is selected to ensure that the software is producing its output or maintaining its behavior as “expected.” The selected parameter value for one test execution is called aTest Case, and the set of test cases selected for testing a system is called aTest Suite. The essence of software testing, therefore, lies in effective ways of identifying the test cases and building the test suite. Two overarching questions related to this process include: 1) how to select the test cases and 2) how to decide when enough test cases have been selected. Over the years, researchers have proposed differentTest Adequacycriteria for answering these two questions.