An empirical study about the effectiveness of debugging when random test cases are used
Mariano Ceccato, Ro Marchetto, Fondazione Bruno Kessler, Leonardo Mariani, Cu Duy Nguyen, Paolo Tonella, Fondazione Bruno Kessler · 2012
Abstract—Automatically generated test cases are usually evaluated in terms of their fault revealing or coverage capabil-ity. Beside these two aspects, test cases are also the major source of information for fault localization and fixing. The impact of automatically generated test cases on the debugging activity, compared to the use of manually written test cases, has never been studied before. In this paper we report the results obtained from two con-trolled experiments with human subjects performing debugging tasks using automatically generated or manually written test cases. We investigate whether the features of the former type of test cases, which make them less readable and understandable (e.g., unclear test scenarios, meaningless identifiers), have an impact on accuracy and efficiency of debugging. The empirical study is aimed at investigating whether, despite the lack of readability in automatically generated test cases, subjects can still take advantage of them during debugging.