Time-Space Efficient Regression Testing for Configurable Systems
Sabrina Souto, Marcelo d’Amorim · arXiv (Cornell University) · 2017
Configurable systems are those that can be adapted from a set of input options, reflected in code in form of variations. Testing these systems is challenging because of the vast array of configuration possibilities where bugs can hide. In the context of evolution, testing becomes even more challenging — not only code but also the set of plausible configurations can change across versions. This paper proposes EvoSPLat, a regression testing technique for configurable systems that explores all dynamically reachable configurations from a test. EvoSPLat supports two important application scenarios of regression testing. In the RCS scenario EvoSPLat prunes configurations (not tests) that are not impacted by changes. In the RTS scenario EvoSPLat prunes tests (not configurations) which are not impacted by changes. To evaluate EvoSPLat under the RCS scenario we used a selection of configurable Java programs. Results indicate that EvoSPLat reduced time by ∼22% and reduced the number of configurations tested by ∼45%. To evaluate EvoSPLat under the RTS scenario we used GCC. Results indicate that EvoSPLat reduced time to run tests by ∼35%. Overall, results suggest that EvoSPLat is a promising technique to test configurable systems in the prevalent scenario of evolution.