Using requirements decision points to identify system test cases: fam test extraction technique and applications
Lee White, Robert Louis Vanderwall · 2003
This dissertation presents a technique for extracting test cases from a requirements document. Although many techniques already exist to do just that, this dissertation describes the Factor-Action Matrix (FAM) technique and the idea of requirements decision points upon which the FAM technique relies. A theoretical model is presented in which these requirements decision points are defined. Using the notion of decision points, the FAM technique is formally developed and the use of the FAM technique in practice is described. This formal description of the technique has aided in both the understanding and the teaching of the technique. In addition, two of the primary claims made by practitioners of the FAM technique are experimentally corroborated. Twenty-one industrial projects that utilize some form of FAM-like test case extraction constitute the case studies used in this work. The analysis of the data collected from these studies was compared with experiential expectations as well as the predictions of the formalization of FAM technique. A particularly interesting observation was made in these industrial studies, that the graph of faults as a function of test case execution showed a surprisingly linear characteristic. Several experiments were run on student projects in which the requirements, testing and development processes were kept under strict control. The results of these experiments are used to better clarify the observations made in the industrial studies. The linearity observation has applications in providing a control line for project progress. An error model has been defined based on the idea of decision points and a simulation was created to explore this model. The simulations exhibited the observed linear behavior and provided insights into the effects of the various parameters. When parameters from industrial projects were used in the simulations, the results coincided with the actual data. With the insight gained from the discussion of decision points, the error model, and the simulations, the behavior of the industrial case studies as well as the student experiments is explained. Additionally, a clear understanding of when the model is applicable emerged. The dissertation concludes with potential extensions to the FAM technique and outlines for future research.