Towards More Precise Coincidental Correctness Detection With Deep Semantic Learning
Huan Xie, Yan Lei, Meng Ling Yan, Shanshan Li, Xiaoguang Mao, Yue Yu, David Lo · IEEE Transactions on Software Engineering · 2024
Coincidental correctness (CC) is a situation during the execution of a test case, the buggy entity is executed, but the program behaves correctly as expected. Many automated fault localization (FL) techniques use runtime information to discover the underlying connection between the executed buggy entity and the failing test result. The existence of CC will weaken such connection, mislead the FL algorithms to build inaccurate models, and consequently, decrease the localization accuracy. To alleviate the adverse effect of CC on FL, CC detection techniques have been proposed to identify the possible CC tests via heuristic or machine learning algorithms. However, their performance on precision is not satisfactory since they overestimate the possible CC tests and are insufficient in learning the deep semantic features. In this work, we propose a novelTriplet network-basedCoincidentalCorrectness detection technique (i.e.,TriCoCo) to overcome the limitations of the prior works.TriCoConarrows the possible CC tests by designing three features to identify genuine passing tests. Instead of using all tests as inputs by existing techniques,TriCoCotakes the identified genuine passing tests and failing ones to train a triplet model that can evaluate their relative distance. Finally,TriCoCoinfers the probability of being a CC test of the test in the rest of the passing tests by using the trained triplet model. We conduct large-scale experiments to evaluateTriCoCobased on the widely-used Defects4J benchmark. The results demonstrate thatTriCoCocan improve not only the precision of CC detection but also the effectiveness of FL techniques,e.g.,the precision ofTriCoCois 80.33$\%$on average, andTriCoCoboosts the efficacy of DStar by 18$\%$–74$\%$in terms of MFR metric when compared to seven state-of-the-art CC detection baselines.