Improved Metrics for Non-Classic Test Prioritization Problems
Ziyuan Wang · Proceedings/Proceedings of the ... International Conference on Software Engineering and Knowledge Engineering · 2015
The average percent of faults detected (AP F D) and its variant versions are widely used to evaluate prioritized test suite's efficiency.However, AP F D is only available for classic test case prioritization, where all prioritized test suites under comparison contain the same number of test cases.If people overlook this phenomenon, they may obtain incorrect results in some non-classic scenarios, where prioritized test suites have different sizes.In addition, it can't precisely illustrate the process of fault detection.Besides the AP F D, most of its variants have similar problems.This paper points out these limitations in detail, by analyzing the physical explanation of AP F D series metrics.To avoid limitations, a series of metrics including RAP F D, RAP F DC , RAP F DW and RAP F DCW are proposed for different types of scenarios.All proposed metrics refer to both the speed of fault detection and the constraint of testing resource.There is an example in this paper showing that proposed metrics provide much more precise illustrations of fault detection process and fault detection efficiency of test suite.