Towards test suite optimization in software component testing by finding and analysing repeats in test trace
Roope Kallio · 2015
Regression testing is often a very resource and time consuming activity with large scale software. However, regression tests may be run in continuous integration where everything should run fast in order to get quick feedback. This thesis reviews the existing solutions to optimize regression test suites and test suites in general by looking at various existing heuristic, computational intelligence and other methods. In most cases, these methods are based on the fact that test suites may contain redundant test cases. They also require code coverage, fault detection capability or requirement coverage information of each test case in order to be implemented. Additionally, the thesis studies ways to detect redundancy in test trace. It is also discussed whether or not this kind of redundancy information can be used for test suite optimization instead. A new concept, pair-wise supermaximal repeats is presented. A case study is conducted in software component continuous integration testing environment in Nokia Networks. Main methods include looking for patterns in test trace manually in a plot representing the test trace as well as finding pair-wise supermaximal repeats in the test trace with an enhanced suffix array implementation developed in test driven development method. A new application was developed, MRS Finder. The repeats are then analyzed. The study shows that the there can be found huge amount of repetition which makes analyzing all of it hard by hand. The repeats also tend to occur more in the setup or teardown phase of a test case rather than in the action or assertion phase of the test cases. The study shows that MRS Finder offers different optimization possibilities than originally was thought.