The Darker Side of Metrics
Douglas C. Hoffman · 2000
There sometimes is a decidedly dark side to software metrics that many of us have observed, but few have openly discussed. It is clear to me that we often get what we ask for with software metrics and we sometimes get side effects from the metrics that overshadow any value we might derive from the metrics information. Whether or not our models are correct, and regardless of how well or poorly we collect and compute software metrics, people’s behaviors change in predictable ways to provide the answers management asks for when metrics are applied. I believe most people in this field are hard working and well intentioned, and even though some of the behaviors caused by metrics may seem strange, odd, or even silly, they are serious responses created in organizations because of the use of metrics. Some of these actions seriously hamper productivity and can effectively reduce quality. This paper focuses on a metric that I’ve seen used in many organizations (readiness for release) and some of the disruptive results in those organizations. I’ve focused on three different metrics that have been used and a few examples of the behaviors elicited in organizations using the metrics. For obvious reasons, the examples have been altered to protect the innocent (or guilty). Biography Douglas Hoffman is an independent consultant with Software Quality Methods, LLC. He has been in the software engineering and quality assurance fields for over 25 years and now is a management consultant specializing in strategic and tactical planning for software quality. He is Section Chairman for the Santa Clara Valley Section of the American Society for Quality (ASQ) and is past Chairman of the Silicon Valley Software Quality Association (SSQA). He is also a member of the ACM and IEEE, and is certificated by ASQ in Software Quality Engineering and has been a registered ISO 9000 Lead Auditor. He has earned an MBA as well as an MS in Electrical Engineering and BA in Computer Science. He has been a speaker at dozens of software quality conferences including PNSQC and has been Program Chairman for several international conferences on software quality.