Expert Estimation of Web-development Effort : Individual Biases and Group Processes
Kjetil Moløkken · NORA - Norwegian Open Research Archives · 2002
Frequently, effort estimates of software development projects are based on expert estimation , i.e., human judgment without much support from documented data of previous projects or estimation models. In many cases this approach may be just as accurate, or even more accurate, compared with formal estimation models. The experts are, however, more prone to systematic biases, e.g., to be impacted by irrelevant information or to follow other goals than accuracy. This thesis investigates the impact of possible biases and shortcomings of expert estimation. An analysis of web-development project data and results from an experiment suggest that: (1) Experts with high technical competence are more optimistic than experts with less technical competence. (2) The reported prediction intervals, i.e., minimum-maximum effort intervals, are too narrow to reflect a high degree of certainty. (3) Experts specifying narrow prediction intervals or very fine-grained estimates do not have higher estimation accuracy. Possible reasons for these biases and shortcomings are (a) Choice of strategy: Estimators with technical background tend to use a bottom-up estimation approach. This approach may easily lead to left-out tasks, and may cause unexpected events to be underestimated. Employing a top-down strategy prevents this by comparing the complete project with other similar projects. (b) Evaluation: How employees are evaluated may explain differences between experts in different roles. Experts in technical roles tend to believe that they are rated as more skilled when delivering optimistic estimates. The same phenomenon may also explain why there is a general preference for narrow prediction intervals. An additional result from the experiment was that: (4) Group discussion based expert estimates tend to be less optimistic than the individuals estimates, i.e. the use of estimation groups may reduce some of the individual biases. The relative widths of the intervals did not increase in the group based effort estimate. A possible explanation is that the uncertainty related to each activity is rated equally in both individual and group settings, but that more tasks are included in the group decision.