Probability Model/Operational Profile Testing
Michael E. Dyer · Encyclopedia of Software Engineering · 2002
Abstract Software test is the process of executing software products against a set of defined inputs to verify that product execution results in a set of predefined outputs. During a product development, software test is typically performed at different times, by different people, and for different reasons. When the software product is being developed, the product developer will always test the software. After the developer is satisfied with the product's capability and quality, more often than not, the software will also undergo testing by an independent test organization or the product customers. In general, the focus of software test takes two directions. Structural test is performed primarily during software development with the goal of verifying that the software, as implemented, matches its design. Functional test is performed primarily by independent testers and/or customers with the goal of verifying that the product design and its software implementation deliver the customer required capability. This article is targeted to the subject of software functional test and, in particular, to use of operational profiles for the conduct of functional testing. Functional testing is performed through a systematic process of (1) identifying product characteristics, as defined by requirements to be verified, (2) creating appropriate test materials (inputs, environments, expected outputs), (3) executing the software using those materials, and (4) reporting the results of the test executions. A functional test plan must be created to identify which of the specified requirements are to be verified and by what verification method. The more common challenge facing the functional tester is how best to select or sample the product requirements and input data to organize test procedures that verify the software to an acceptable level of confidence. In the current research on test formulation, two distinct approaches have been defined that can loosely be classified as deterministic and statistical. In the deterministic approach, test procedures are organized through a systematic analysis of the requirements specification and/or software implementation. A variety of domain partitioning and data flow analysis techniques have been defined to support the deterministic method. In the statistical approach, test procedures are organized through a random selection process that is based on probability distributions defined across the product usage domain. Available surveys of methods for functional testing tend to indicate that preponderance of research into testing methods has focused on the deterministic approach. This article attempts to provide more insight into statistically based functional test methods and specifically addresses the functional testing methods that use operational profiles for organizing test samples.