Regression testing with GZoltar techniques for test suite minimization, selection, and prioritization
José Carlos de Campos · 2012
Software testing occurs simultaneously during the software development to detect errors as early as possible and to guarantee that changes made in software did not affect the system negatively. However, during the development phase, the test suite is updated and tends to increase in size. Due to the resource and time constraints for re-executing large test suites, it is important to develop techniques to reduce the effort of regression testing. Several approaches have been studied to reduce the effort of regression testing: test suite minimization, selection, and prioritization. Test suite minimization techniques aims at identifying and eliminating redundant test cases from the suite. Test suite selection techniques identifies a subset of test cases from suite, required to re-test the changes in the software. Test suite prioritization techniques schedule test cases for execution in an order to increase the early fault detection. Although, the approaches present the literature have some limitations (resources, times, etc.). In this manuscript we propose a new approach to the minimization, selection and prioritization problem that try to overcomes these limitation. Our approach allows for (1) easy encoding of the relation between a test case and a component (source code statement) in a coverage matrix, (2) mapping this matrix in a set of constraints, and (3) computing optimal solutions (with the same coverage and fault detection of original set of tests) to minimization and selection problems by leveraging a fast and scalable constraint solver. We implemented our approach in a Eclipse plug-in called GZOLTAR, which is perfectly integrated in the IDE and provides functions to debugging a software program using visualization in OpenGL. Features for regression tests were implemented with the name Regression-Zoltar, RZOLTAR for short. To test our toolset, we conducted experiments to measure the efficiency and the size of regression suite. In particular, we evaluated our tool using real-world and large programs, as well as a user study in order to measure its usability. In our empirical evaluation, we show how RZOLTAR can be used to instantiate large-programs and efficiently find an solution for such problems. Our experiments show that our approach can significantly reduce the size of original test suite, and a corresponding reduction in the cost of regression testing. We also make a user study to find if we are in right way in terms of usability and functionalities.