On the testing of user‐configurable software systems using firewalls
Brian Robinson, Lee White · Software Testing Verification and Reliability · 2010
SUMMARY User‐configurable software systems present many challenges to software testers. These systems are created to address a large number of possible uses, each of which is based on a specific configuration. As configurations are made up of groups of configurable elements and settings, a huge number of possible combinations exist. As it is infeasible to test all configurations before release, many latent defects remain in the software once deployed. A just‐in‐time testing technique is presented to address this problem, including examples of how it can be used with various user‐configurable systems in the field. The proposed solution is evaluated with a set of five empirical studies conducted on two separate ABB software systems using real customer configurations and changes. These studies analyse failures reported by many different customers globally and show that this just‐in‐time testing process is effective at detecting latent defects exposed by customer configuration changes in user‐configurable systems. Copyright © 2010 John Wiley & Sons, Ltd.