Clarifications on the Construction and Use of the ManyBugs Benchmark
Claire Le Goues, Yuriy Brun, Stephanie Forrest, Westley R. Weimer · IEEE Transactions on Software Engineering · 2017
Automated repair techniques produce variant php interpreters, which should naturally serve as the tested interpreters. However, the answer to the question of what should serve as the testing interpreter is less obvious. php's default test harness configuration uses the same version of the interpreter for both the tested and testing interpreter. However, php may be configured via a command-line argument to use a different interpreter, such as the unmodified defective version, or a separate, manually-repaired version.