Editorial: A test of time across the generations
Martin Woodward · Software Testing Verification and Reliability · 2004
A test of time across the generationsFundamental to the process of software testing is the generation of appropriate test data.Both of the papers in this issue address this topic and, indeed, both are concerned with automating the activity.The word 'generation' has a number of related meanings, but here, of course, it is being used in the sense of 'production' or 'creation'.Legeard et al. consider the automated generation of test cases from formal models of systems in the B abstract machine notation in their paper.In fact, their approach utilizes a system called BZ-TESTING-TOOLS (BZ-TT), which can be used for test case generation from Z specifications as well as B abstract machines.The focus of the paper is on the definition of control-flow and data-oriented coverage criteria for B abstract machines.The data-oriented criteria are particularly interesting since the authors have developed a family of criteria, the aim of which is to test the boundaries of a given state space.In a manner similar to domain testing, the underlying assumption is that faults are more likely to be detected on the boundaries of input domains than in the interiors.A nice feature of the BZ-TT environment is that it allows the systematic generation of a small number of test cases that provide good coverage, whilst also allowing the test engineer to expand the coverage of specific parts of the model that are particular areas of concern.In this way, the problem of an explosion in the number of test cases generated can be carefully controlled and kept manageable, using the variety of well-defined coverage criteria.The approach has been applied to a number of industrial case studies and brief details of one of these, a smart card system, are given in the paper.McMinn's paper is rather different in that it provides a survey of metaheuristic search techniques for the automatic generation of test data.In any area of research it can be useful to pause and take stock of what has been achieved.It also gives an opportunity to reflect on outstanding problems and possible future directions.McMinn defines metaheuristic search techniques as 'high-level frameworks, which utilize heuristics to seek solutions for combinatorial problems at reasonable computational cost'.One such technique is simulated annealing, which gets its name by analogy with the chemical process of annealing-heating a solid until it melts and then changing its properties by varying the rate at which it cools back to a solid.Another technique involves genetic algorithms (GAs), an approach inspired by genetics and Darwinian natural selection.GAs bring into play a second meaning of the word 'generation', namely descendants of some given population in the same number of steps.Starting from an initial population of solutions, the fittest individuals undergo iterative recombination and mutation (somewhat akin to breeding), thereby evolving successive generations.Metaheuristic search techniques, such as the two just highlighted, have been used for automatic test data generation:• to obtain coverage of specific program structures, e.g.statements or branches;• to exercise specific program features as described by a specification;• to attempt the disproving of certain properties, e.g.finding test data to falsify an assertion;• to verify non-functional properties, e.g.worst-case execution times.McMinn provides a very clear exposition of work in all these areas, finishing each section with helpful suggestions for further research.