A study of complexity metrics as surrogate measures of software maintainability
Virginia R. Gibson · 1986
The purpose of this research was to investigate the relationship between system complexity and system maintainability. Complexity was measured by six automatable metrics: Halstead's Effort (E), McCabe's Cyclomatic complexity (V(g)), Woodward's Knot measure (K), Chen's Maximal Intersect Number (MIN), Gaffney's Unconditional Jumps (J), and Benyon-Tinker's depth and breadth measure (C(x)). These metrics do not provide consistent rankings of relative system complexity. When applied to three versions of a 2,000-line COBOL file management system, decreases in E, V(g), K, and J were associated with increases in MIN and C(x). System maintainability was measured by performance. Each of thirty-six experienced programmers performed three perfective maintenance tasks. A 3 x 3 randomized-block, repeated-measures, partially-confounded experimental design was used. The time taken to implement changes, the accuracy of the modifications, programmers' confidence in the correctness of their work, and programmers' perceptions of relative system complexity were recorded. Results indicate that complexity metrics offer potential for measuring maintainability of COBOL systems. Although the metrics were not related to performance on individual tasks, across the portfolio of three diverse tasks, strong relationships were observed. Metrics which assess bulk and control flow complexity (E, V(g), K, and J) were related to maintenance time and ripple effect error frequency. Metrics which reflect depth of IF nesting and calling hierarchies (MIN and C(x)) were related to primary error frequency. The bulk and control flow metrics appear to be the more important measures. These metrics are related to both maintenance time and frequency of ripple effect errors. Since ripple effect errors are more difficult to uncover than primary errors, it is important to reduce the likelihood of their occurrence. The portfolio effect suggests that the metrics offer better potential for assessing relative lifetime costs than for assessing the costs of individual maintenance tasks.