Testing and integration strategies of large software systems
Jeffery Arthur Solheim · 1992
In the field of software engineering, there has been much discussion about the merits of various testing and integration strategies. Top-down, bottom-up, big-bang, and sandwich integration strategies are advocated by various authors. Also, some authors insist that modules be unit tested, while other authors feel that unit testing can lead to basic system errors. This dissertation addresses the ability of the aforementioned integration strategies to detect and correct defects, and produce reliable systems. This dissertation also explores the efficacy of unit testing when used in combination with the aforementioned strategies. The research summarized in this dissertation employed artificial software systems - artificial in the sense that each module of a system performed a simple computation. Programs automatically generated the Ada implementations of two versions of each system - a correct version (containing no defective modules), and a defective version (seeded with common software defects). Other programs implemented unit test and the aforementioned integration strategies. These programs were applied to the defective version of each system. For each test and integration strategy, data were gathered which measured the defect-correction ability and resulting system reliability. Because of the great execution time required to test and integrate systems, a system simulator was developed which simulated the actions of both the correct and defective versions of a system. Experimental results indicated that top-down integration strategies are most effective in terms of defect correction, but also exercise the greatest number of modules. Bottom-up strategies are generally least effective at correcting defects, but exercise relatively few modules. Results also indicated that the top-down and big-bang strategies produce the most reliable systems, and the bottom-up strategies produce the least reliable systems. Systems integrated by the sandwich strategy are moderately reliable, in comparison. Experimental results favored neither those strategies that incorporate unit testing nor those that do not. Also, results favored neither phased nor incremental strategies.