THE LAST 9% UNCOVERED BLOCKS OF CODE - A CASE STUDY
Cristina Daniela Manu, Pooja Nagpal, Donny Amalo, Roy Patrick Tan · 2010
Code coverage is an efficient technique for understanding what code has been exercised by an existing test bed, yet it is often debated how much effort should be invested in increasing coverage. This paper is a case study of our code coverage effort during the product development cycle for a component of the .NET Framework 4. At the end of the cycle, we had a focused initiative to increase the code coverage from 91% to 100%. In order to measure the effectiveness of increasing code coverage, we devised a few metrics based on the number and importance of the bugs found, the time invested, the increase in code coverage and bug yield in comparison with other test activities. Our results showed that: 1. It is not prohibitively expensive to achieve effective 1 -100% code coverage. 2. There is no significant increase in the number of bugs found with higher code coverage. In this paper, we discuss what parts of our development process (such as the propensity of testers to develop more complex scenarios first) may have contributed to this lack of bugs.