FLITSR: Improved Spectrum-Based Localization of Multiple Faults by Iterative Test Suite Reduction
Dylan Callaghan, Bernd Fischer · ACM Transactions on Software Engineering and Methodology · 2025
Spectrum-based fault localization (SBFL) works well for single-fault programs but its accuracy decays for increasing fault numbers. We present FLITSR (Fault Localization by Iterative Test Suite Reduction), a novel SBFL approach that improves the localization of a given SBFL base metric specifically in the presence of multiple faults. FLITSR iteratively selects reduced versions of the test suite that better localize the individual faults in the system. This allows it to identify and re-rank faults ranked too low by the base metric because they were masked by other program elements. Through this process, FLITSR returns a set of highly suspicious program elements (called a basis), where the execution of each failing test involves at least one basis element, considered as the cause of the failure. We implemented the FLITSR algorithm in an open source toolset and extensively evaluated it over three true multi-fault datasets, varying the fault type, coverage granularity and programming language. Our evaluation shows that FLITSR consistently outperforms existing localization metrics and methods, including those designed to handle multiple faults such as ARTEMIS and parallel debugging. For the Defects4J method-level faults, FLITSR also substantially outperforms GRACE, a state-of-the-art learning-based fault localizer.