Strategies to Automatically Test Eiel Programs
Andreas Leitner · 2004
Semi-automated software testing has become an increasingly popular and effective tool for raising the quality of software. Pre-, postconditions, and invariants describe the semantics of a feature (i.e., method or data member) and can help to decide whether a given test failed or passed [ARM03, Gre04, Ciu04]. Since the decision can be made without further knowledge of the input this opens up the possibility of a fully automated push button tester. Implementing such a system reveals both theoretical and practical challenges. A fully automated tester is likely to run the system under test into unrecoverable states and must at least be able to recover gracefully in order to continue testing. A master/slave mechanism has been implemented to automatically resume testing after a runtime crash. The tester must be able to satisfy the preconditions of the features under test in order to call them. Automatically satisfying preconditions in Eiffel is a challenging problem, since the contract is in practice never complete. Even if it is, its expression language is both non-functional and undecidable. Using the very simple approach of (almost) random input data yields promising results, but features with strong preconditions are left untested. A more guided approach based on a planning system has been implemented for those cases. From Eiffel source code a planning problem is generated with the goal being the precondition of the feature under test. A planner is used to generate a plan, which is then executed by an Eiffel interpreter. If the goal can be reached, the precondition of the feature under test is satisfied and thus the feature can now be tested, otherwise information from the plan execution is used to augment the planning problem and the planner is asked for a new plan.