Extending the Boundaries in Regression Testing: Complexity, Latency, and Expertise
Shin Yoo · UCL Discovery (University College London) · 2009
Automated test case management techniques have been studied in order to aid regression testing tasks by reducing their cost and improving their efficiency. However, the current state-of-art test case management techniques remain limited in several aspects. First, the existing techniques share a scalability problem, not only in terms of the size of the test suites and SUT (System Under Test), but also in terms of complexity of the constraints expressible in the process of regression testing, because the problem is often formulated without considering additional constraints. Second, there is no guarantee that the reduced effort does not compromise the fault detection capability of the test suite. Finally, the existing techniques do not provide any means for the human testers to contribute their important domain knowledge. This domain knowledge is hard to capture algorithmically. This thesis aims to reformulate test case management techniques in these regards by presenting new concepts, algorithms, approaches and combinations of techniques. In order to deal with the complexity of real world regression testing, multi-objective formulation of test case management is presented, allowing the human tester to apply test case management techniques while meeting to multiple objectives. The thesis introduces the concept of latency, which is used to measure the redundancy in a test suite systematically so that the tester can make an informed decision on the appropriate size of test suites. It also shows that the latency of a test suite can be improved automatically using search-based test data augmentation techniques, which can be significantly more efficient compared to existing test data generation techniques. Finally, the thesis considers the combination of clustering and pair-wise comparison approaches to efficiently incorporate the domain knowledge of human tester into test case management.