A comparison of cross-company and within-company effort estimation models for Web applications
Barbara Ann Kitchenham · 2004
The objective of this paper is to investigate the use of cross-company and within-company cost estimation model for Web projects. The study analysis method we used was a forward, stepwise regression of Web project development effort with independent variables that were used for on-line Web application price quotations. Model cross-validation used two approaches: one-fold cross validation and a validation subset of 13 projects from the same company. The setting consists of Industrial Web applications from 24 different companies and 8 different countries, and the experiment units are 53 Web application and hypermedia projects. The main outcome measure was an estimate of effort for each project, for each estimation model. We used the summary statistics: MMRE, Median MRE, Pred(25), mean and median of absolute residuals to evaluate the predictive accuracy of model. We also compared estimation model accuracy with an assessment of estimate accuracy provided by the participating companies. Results showed that the best fitting cross-company regression model was significantly better than the median and experts at predicting effort for 13 projects from a single company (p < 0.05). The best fitting within-company regression model for the company with 13 projects was significantly better than the cross-company model (p < 0.05). However the accuracy of the company 1 model was worse than the accuracy achieved by company 1 estimators. In addition, the company 1 model was an extremely poor predictor of effort for the other companies' projects. We conclude that cross-company effort estimation model can be useful for companies that do not have past projects from which to develop their own models. However, a cross-company data set should only be used for estimation until it is possible for a Web company to build its own within-company model. For cross-company and within company models, simple measures, available early in the lifecycle, that can be estimated by clients are suitable for building effort estimation models.