Inadequate Testing, Time Pressure, and (Over) Confidence: A Tale of Continuous Integration Users
Gustavo Pinto, Marcel Rebouças, Fernando Castor · 2017
Continuous Integration (CI) prescribes that changes should be integrated into the main codebase as often as possible and that the system should be built frequently. To support CI, a number of software tools have been developed. However, little is known about the main reasons for build breakage and whether CI delivers its promise of early problem detection and smooth integration. To shed light on this issue, we conducted a survey with 158 CI users. Through a qualitative investigation we found that inadequate testing is the most common technical reason related to build breakage, whereas lack of time plays a role on the social reasons. Still, although some respondents reported that CI usage increases the confidence that the code is in a known state, some respondents also reported that there is a false sense of confidence when blindly trusting tests.