The Application of High-Throughput Testing and Design of Experiments to Cyber Systems
Gaurav Simon Gandhi · The UNSW Canberra at ADFA Journal of Undergraduate Engineering Research · 2018
The cyber space has been expanding rapidly in recent years, and with this increase comes the necessity for continual improvement in the test and evaluation of cyber systems. Scientific exploration can be used to gain a better understanding of the space, allowing us to better stop or defend against cyber-attacks. However, to date, there appears to be a lack of published research that has been conducted for testing in the field, and of the research reviewed, even less have used a proper experimental methodology, or a systematic analysis that could provide rigorous results. High-Throughput Testing uses combinatorial methods to reduce the number of test cases in an experiment, whilst ensuring a predetermined coverage level. Combined with Design of Experiments, which allows us to determine the effects of each individual input factor in a system, and interactions between factors which often are not intuitive, we can design efficient and effective experiments that allow us to better understand the system under study. These techniques are already being employed extensively by the U.S. military, to improve test and evaluation procedures. This project was conducted to determine the applicability of High-Throughput Testing and Design of Experiments techniques to testing a simple cyber system, whilst following a proper experimental method, so as to provide the framework for future studies into more complex systems.