Do Mutations of Strongly Subsuming Second-Order Mutants Really Mask Each Other?
João Paulo Diniz, Fischer Ferreira, Fabiano Cutigi Ferrari, Eduardo Figueiredo · 2023
Mutation testing is a fault-based testing criterion that is used to measure the quality of the test suites of software systems. Due to its inherent high computational cost, many studies were published in the last decades aiming at reducing computational cost and human-effort for the mutation analysis. One of the most promising areas is searching for Strongly Subsuming Higher-Order Mutants (SSHOMs), which are rare and harder to kill than their constituent first-order mutants (FOMs). Therefore, they are valuable especially because they can replace their FOMs without loss of effectiveness in the mutation testing process. One of the explanations for the SSHOMs to be harder to kill than their constituent FOMs is that the single faults (mutations) can partially mask one another, so that the combination of them is harder to detect than any of the individual faults. However, we did not find in the literature an investigation of the masking phenomenon. Therefore, the goal of this paper is to start filling this gap. More specifically, for a mutation to mask the other one, it is necessary firstly that the execution of a test case reaches all involved mutations. Therefore, we designed two complementary studies to accomplish our goal. Study #1 focuses on searching for Strongly-Subsuming Second-Order Mutants (SS2OMs) and then analyzes reaching characteristics of their constituent FOMs. We found that almost half of the SS2OMs constituent FOMs are not even reaching the other FOM. In Study #2, we designed a search strategy that considers a second-order mutant killed only if both of their mutations are reached by a failing test case execution. This strategy found much more SS2OMs than in the first study.