Study of the relationship of bug consistency with respect to performance of spectra metrics
Hua Jie Lee, Lee Naish, Kotagiri Ramamohanarao · 2009
In practice, manual debugging to locate bugs is a daunting and time-consuming task. By using software fault localization, we can reduce this time substantially. The technique of software fault localization can be performed using execution profiles of the software under several test inputs. Such profiles, known as program spectra, consist of the coverage of correct and incorrect executions statement from a given test suite. We have performed a systematic evaluation of several metrics that make use of measurement obtained from program spectra on Siemens Test Suite. In this paper, we discuss how the effectiveness of various metrics degrade in determining buggy statements as the bug consistency (error detection accuracy, qe) of a statement approaches zero. Bug consistency of a statement refers to the ratio of the number of failed tests executing the statement over the total number of tests executing the statement. We proposed Effect(M) as to measure the effectiveness of these metrics as qevalue varies. We also demonstrate that the qe(previously not considered as a metric), is just as effective as some of the metrics proposed. We also formally prove that qeis identical to the metric that Tarantula system uses for bug localization.