Writing Maintainable Automated Acceptance Tests
Dale H. Emery · 2009
Test automation is software development 1 . This principle implies that much of what we know about writing software also applies to test automation. And some of the things we know may not be apparent to people with little or no experience writing software. Much of the cost of software development is maintenance—changing the software after it is written. This single fact accounts for much of the difference between successful and unsuccessful test automation efforts. I’ve talked to people in many organizations that attempted test automation only to abandon the effort within a few months. When I ask what led them to abandon test automation, the most common answer is that the tests quickly became brittle and too costly to maintain. The slightest change in the implementation of the system—for example, renaming a button—breaks swarms of tests, and fixing the tests is too time consuming. But some organizations succeed with test automation. Don’t they experience maintenance costs, too? Of course they do. An important difference is that where unsuccessful organizations are surprised by the maintenance costs, successful organizations expect them. The difference between success and failure is not the maintenance costs per se, but whether the organization expects them. Successful organizations understand that test automation is software development, that it involves significant maintenance costs, and that they can and must make deliberate, vigilant effort to keep maintenance costs low. The need to change tests comes from two directions: changes in requirements and changes in the system’s implementation. Either kind of change can break any number of automated tests. If the tests become out of sync with either the requirements or the implementation, people stop running the tests or stop trusting the results. To get the tests back in sync, we must change the tests to adapt to the new requirements or the new implementation.