Test Adequacy Criteria for Metamorphic Testing
Yanwen Liu, Ruifeng Li, Hao Tao, Zheng Zheng · 2024
The Oracle problem poses a fundamental challenge in software testing, seeking to validate the accuracy of a program's output given specific test inputs. Metamorphic testing (MT) has emerged as an automated technique to tackle the Oracle problem by testing programs against metamorphic relations (MRs) - crucial properties that persist across multiple executions of the target program. An essential aspect of software testing involves defining objective metrics to gauge the violation criteria. While many studies concentrate on the defect monitoring capability of MRs, the adequacy criteria for MT often lack coverage information regarding MRs. In this paper, we proposed MTcoverage, an adequacy criterion for MT. MTcoverage has integrated the degree of MR coverage into the adequacy criteria. We employed this criterion to formulate test case generation strategies. The effectiveness of this coverage criterion was also demonstrated by means of mutation testing. The results of this study contribute to enhancing the effectiveness of MT techniques for testing the program under examination.