A firewall model for testing user-configurable software systems
Lee White, Brian P. Robinson · OhioLink ETD Center (Ohio Library and Information Network) · 2008
User-configurable software systems present many challenges to software testers.These systems are created to address a large number of possible uses, each of which is based on specific configurations.Configurations are made with combinations of configurable elements and settings, leading to a huge number of possible combinations.Since it is infeasible to test all combinations at release, many latent defects remain in the software once deployed.An incremental testing approach is presented, where each customer configuration change requires impact analysis and retesting.This incremental approach involves cooperation and communications between the customer and the software vendor.The process for this approach is presented along with detailed examples of how it can be used on various user-configurable systems in the field.The overall efficiency and effectiveness of this method is shown by a set of empirical studies conducted with real customer configuration changes running on two separate commercially released ABB software systems.These two systems together contained ~3000 configurable elements and ~1.4 million Executable Lines of Code.In these five case studies, 460 failures reported by 100 different customers were analyzed.These empirical studies show that this incremental testing method is effective at detecting latent defects which are exposed by customer configuration changes in user-configurable systems.