Prediction of Software Failures Based on Systematic Testing
Michael Grottke · 2001
Software can be tested in several ways: either according to the operational profile of the user, or by following a systematic testing approach. If the latter is chosen, then it becomes difficult to estimate the number of failures in the software, or its reliability. In this paper, we present a software reliability model, the Rivers-Vouk model, which specifically addresses the issues of systematic testing. Using an intuitive model framework its assumptions are compared to those of a model designed for operational testing. The Rivers-Vouk model has been applied to a test project at imbus AG and the results of the failure prediction are discussed. In the conclusion the PETS project is briefly reviewed whose aim it is to incorporate data on the software process maturity into software reliability models. 1 Software testing and failure models No piece of software, independent of its size and complexity, is free of faults. As software is written by humans, errors will always occur. Due to a wrong or incomplete specification, large problem complexity, lack of time, etc. mistakes are made, and when this happens whilst developing software this is known as an error. The result of a human error being made is a software fault, i.e. an incorrect piece of software. When the faulty software is executed, it can exhibit an unexpected behavior, produce an incorrect result. This is known as a failure. Software failures are a serious matter. In a growing and increasingly networked information society, in which the computer has conquered almost every aspect of our lives, a software failure may result in an threat to life or assets. In the Therac-25 accident, for example, several patients lost their lives due to a radiation overdose. Statistics on software failures published in literature clearly document the risks [16]. In this situation it is of vital interest that we thoroughly test critical software and are able to predict the number of failures experienced in the remaining test time or the number of faults in the software after release. With this goal in mind numerous statistical models have been developed. The basic approach is to model past failure data to predict future behavior. These models are typically based on failure data such as the number of failures experienced in specified time intervals, or the time elapsed between two consecutive failures. Reflecting the definition of a failure, it becomes clear that the occurrence of a failure depends on the characteristics of the software and on the way in which the software is used. One possible way to test software is according to the typical behavior of the user, which is also called the operational profile. Testing according to the operational profile means that the operations of the system under test have to be determined together with their execution probabilities. Based on these probabilities the number of test cases is allocated to the different program functions. The test cases should then be executed in a random order, i.e. different parts of the software should not be tested in strict succession. Operational testing has two advantages. Since the operations that will be used frequently by the users are tested more thoroughly, more failures will occur in these parts of the software during testing and consequently these parts will be more stable afterwards (assuming a ∗This work was supported by the European Community in the framework of the specific programme for research, technological development and demonstration on a user friendly society (1998-2002), the “IST Programme”, Contract No. IST-1999-55017. The authors are solely responsible for this paper. It does not represent the opinion of the Community, and the Community is not responsible for any use that might be made of data appearing herein. This article was published in: Electronic Proc. Ninth European Conference on Software Testing Analysis and Review (EuroSTAR), Stockholm, 2001