Simple Plots Improve Software Reliability Prediction Models
John S. Lawson, Craig W. Wesselman, Del T. Scott · Quality Engineering · 2003
Software reliability prediction is accomplished by fitting a nonhomogeneous Poisson process (NHPP) model to data from software testing. The data consist of the cumulative time and the cumulative number of failures found in software testing. The NHPP model can be used to predict the reliability of the software product at the time of release or to determine how much further testing must be done to reach a specified failure rate. Models are normally fitted to software testing data using Poisson regression by the method of maximum likelihood. We encountered difficulties fitting models when numerical algorithms failed to converge or when we were unable to discriminate among several models with the same number of parameters. These difficulties were the result of having no likelihood ratio test to compare models with the same number of parameters and anomalies in the data that caused numerical algorithms to fail. We found that a simple cumulative plot of the data (cumulative failures on the vertical axis vs. cumulative test time on the horizontal axis) helped in spotting anomalies in the data and selecting an appropriate model to fit. A second plot of running products of ratios of the probability densities for the predictions made from competing models, called the prequential likelihood ratio, helped in discriminating between models. Use of these plots helped resolve the difficulties we experienced in fitting models to the software testing data.