Search-based product line fault detection allocating test cases iteratively
Urtzi Markiegi, Aitor Arrieta, Goiuria Sagardui, Leire Etxeberria · 2017
The large number of possible configurations makes it unfeasible to test every single system variant in a product line. Consequently, a small subset of the product line products must be selected, typically following combinatorial interaction testing approaches. Recently, many product line engineering approaches have considered the selection and prioritization of relevant products within the product line. In a further step, these products are thoroughly tested individually. However, the test cases that must be executed in each of the products are not always insignificant, and in systems such as Cyber-Physical System Product Lines (CPSPLs), their test execution time can vary from tens to thousands of seconds. This issue leads to spending a lot of time testing each individual product. To solve this problem we propose a search-based approach to perform the testing of product lines by allocating small number of test cases in each of the products. This approach increases the probability of detecting faults faster. Specifically, our search-based approach obtains a set of products, which are derived from using any state-of-the-art approach as inputs, and a set of attributed test cases. As an output a list of allocated test cases for each product is obtained. We also define a novel fitness function to guide the search and we propose corresponding crossover and mutation operators. The search and test process is iteratively repeated until the time budget is consumed. We performed an evaluation with a CPSPL as a case study. Results suggest that our approach can reduce the fault detection time by 61% and 65% on average when compared with the traditional test process and the Random Search algorithm respectively.