Partial ordering tests by coverage sets
Markus Gaelli, Oscar Nierstrasz, Roel Wuyts · Bern Open Repository and Information System (University of Bern) · 2003
A single software fault may cause several tests to break, if they cover the same methods. The coverage sets of tests may not just overlap, but include one another. This information could be of great use to developers who would like to focus on the most specific test that concerns a given fault. Unfortunately, existing unit testing tools neither gather nor exploit this information. We have developed a simple approach that analyses a set of test suites, and infers the partial order corresponding to inclusion hierarchy of the coverage sets. When several tests in an inclusion chain break, we can guide the developer to the most specific test in the chain. Our first experiments with three case studies suggest that most unit tests for typical applications are, in fact, comparable to other tests, and can therefore be partially ordered. Furthermore, we show that this partial order is semantically meaningful, since faults that cause a test to break will, in nearly all cases cause less specific tests too break too.