Evolutionary Testing for Program Repair
Haifeng Ruan, Hoang Lam Nguyen, Ridwan Shariffdeen, Yannic Noller, Abhik Roychoudhury · 2024
Automated program repair (APR) allows for au-tonomous software protection and improvement. Many proposed repair techniques rely on available test suites, since tests are available in real-world settings. Tests are incomplete specifications, however. As a result, repairs generated based on tests may suffer from the test overfitting problem. The patches produced by APR techniques may pass the given tests and thus be plausible, and yet be an incorrect patch. This hints towards more extensive test suites to guide program repair. Generating additional tests to improve the test suite quality is generally difficult because the oracle or expected observable behavior of the generated tests is unknown. In our work, we first construct additional oracles by instrumenting buggy programs from the Defects4j benchmark with the knowledge obtained from the available bug reports. Then, we formulate a coevolution approach that generates tests and repairs in a unified workflow. The complete workflow is implemented as an extension of the well-known Java testing framework Evosuite. This includes re-purposing the search in Evosuite to search for repairs (instead of searching for tests) and enables an easy adoption for developers who are already familiar with Evosuite for test suite generation. The evaluation of our tool EvoREPAIR shows that coevolution positively impacts the quality of patches and tests. In the future, we hope that such coevolution can inspire new repair tools and techniques.