Testing on a budget
Mohamed Zaït, Allison Lee, Khaled Yagoub, Ravi Sahani, Holly Casaletto, Lokesh Kumar · 2009
Testing complex software is one of the most challenging steps in the software development lifecycle. Several testing methods are usually used from the time the subject software component is ready for testing until the complete system where the software component operates is released for distribution. In this paper, we describe one of the methods used to test the SQL Engine of the Oracle database management system (DBMS). This process has been designed to track, detect and report quality problems along with detailed metrics that make it easier to find the root cause of different types of regressions (failures, result correctness, and performance). The test process has been built on top of a framework that is extensible so that any schema and SQL workload can be plugged into the framework. The process described here is currently part of the Oracle DBMS test process.