Zur Automatisierung von Softwaretests - Entwicklung und Bewertung von Orakellösungen
Ralph Joachim Guderlei · OPen Access Repositorium der Universität Ulm (OPARU) (Ulm University) · 2008
All large software systems contain bugs. Therefore the identification and the removal of these bugs is an important and time consuming task, responsible for a large slice of the software development schedule. The testing of software is the most prominent method to improve software quality. It is part of any modern software development process. Testing is the execution of a software system under defined conditions. The system is monitored during execution and its output is collected. A bug is identified if the behaviour of the program (i.e. its output) does not correspond to its specification. There are two main problems in software testing. The first problem is the choice of the test input, the second problem is the so-called oracle problem. An oracle is a mechanism to obtain reference values and to compare the actual output of the system under test to these reference values. Often, there is no oracle available. In the present thesis, several oracles are presented and discussed. Some of the presented oracles were newly developed. Additionally, a new method for the measurement of the effectiveness of an oracle was developed. This effectiveness measure is based on a well-known test adequacy criterion, the so-called mutation analysis technique. The presented oracle methods were examined using software from open source projects, showing that these methods can be applied to real life software. The experiments lead to the conclusion that the proposed methods are effective. The results of the examination were also used to derive guidelines for the improvement of the effectiveness of the proposed methods. The presented methods can be applied automatically. That means that the test decisions are made without human intervention. Therefore, the test results are reproducible and unbiased. The automation of the testing process also reduces the necessary amount of time for running and rerunning the tests.