Guided Random-Based Testing Strategies Diploma Thesis
Cosmin Mitran · 2007
Software testing is no longer considered a peripherical activity during the software development process. Its importance is widely recognized and researchers invest a lot of efforts in order to make testing less tedious and more effective. Despite this, progresses in this field are not yet advanced enough to offer software developers satisfactory solutions for their testing needs. Manual and automated testing are the two complementary ways in which software can be tested. While manual tests are built according to the testing engineer’s knowledge about the system under test, automated testing relies on the computation power of a machine for performing many more tests than a human could in a limited period of time. So, it is desirable to benefit from the advantages of both methods in order to test any software system effectively and intensively. In spite of this fact, manual and automatic testing tools are usually different, often making the testing process laborious and hard to manage. AutoTest is a testing tool that reconciles the two approaches: it permits both manual and automated tests inside the same testing process; furthermore, it generates, compiles and runs tests on the push of a button. It relies on the principles of Design by Contract for assessing whether a test passes or not. AutoTest has found bugs in several contract equipped Eiffel libraries, proving itself to be a valid solution for the challenge of testing. The main goal of the current project was to implement new strategies for making the automatic testing process more accurate and effective, trying to take advantage of the user’s knowledge contained in manual tests. The automatic tests attempt to reproduce as much as possible the distribution of inputs for the manual tests, while selecting representative test cases from the whole input space. Automatic tests need to be diverse in order to be effective. They also need to be close to the manual tests, which contain information about the system under test. Our strategies try to reconcile these two apparently opposite requirements. Another goal of the current work was to provide the user with various statistics about the testing process: classifications of the causes that lead to failures; the number of failure triggering lines found in the tested code; the amount of time elapsed until the first failure is triggered. Gathered statistics are displayed in an easy to read format.