Studying Test Case Failure Prediction for Test Case Prioritization
Tanzeem Bin Noor, Hadi Hemmati · 2017
Background: Test case prioritization refers to the process of ranking test cases within a test suite for execution. The goal is ranking fault revealing test cases higher so that in case of limited budget one only executes the top ranked tests and still detects as many bugs as possible. Since the actual fault detection ability of test cases is unknown before execution, heuristics such as "code coverage" of the test cases are used for ranking test cases. Other test quality metrics such as "coverage of the changed parts of the code" and "number of fails in the past"' have also been studied in the literature. Aims: In this paper, we propose using a logistic regression model to predict the failing test cases in the current release based on a set of test quality metrics. Method: We have studied the effect of including our newly proposed quality metric ("similarity-based" metric) into this model for tests prioritization. Results: The results of our experiments on five open source systems show that none of the individual quality metrics of our study outperforms the others in all the projects. Conclusions: However, the ranks given by the regression model are more consistent in prioritizing fault revealing test cases in the current release.