Rapid Regression Detection in Software Deployments through Sequential Testing
Michael Lindon, Chris Sanden, Vaché Shirikian · Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining · 2022
The practice of continuous deployment has enabled companies to reduce time-to-market by increasing the rate at which software can be deployed. However, deploying more frequently bears the risk that occasionally defective changes are released. For Internet companies, this has the potential to degrade the user experience and increase user abandonment. Therefore, quality control gates are an important component of the software delivery process. These are used to build confidence in the reliability of a release or change. Towards this end, a common approach is to perform a canary test to evaluate new software under production workloads. Detecting defects as early as possible is necessary to reduce exposure and to provide immediate feedback to the developer.