High coverage testing of Haskell programs
Tristan O.R. Allwood, Cristian Cadar, Susan Eisenbach · 2011
This paper presents a new lightweight technique for auto-matically generating high coverage test suites for Haskell library code. Our approach combines four main features to increase test coverage: (1) automatically inferring the constructors and functions needed to generate test data; (2) using needed narrowing to take advantage of Haskell’s lazy evaluation semantics; (3) inspecting elements inside re-turned data structures through the use of case statements, and (4) efficiently handling polymorphism by lazily instan-tiating all possible instances. We have implemented this technique in Irulan, a fully au-tomatic tool for systematic black-box unit testing of Haskell library code. We have designed Irulan to generate high cov-erage test suites and detect common programming errors in the process. We have applied Irulan to over 50 programs from the spectral and real suites of the nofib benchmark and show that it can effectively generate high-coverage test suites—exhibiting 70.83 % coverage for spectral and 59.78% coverage for real—and find errors in these programs. Our techniques are general enough to be useful for several other types of testing, and we also discuss our experience of using Irulan for property and regression testing.