Combinatorial Testing Tool Learnability in an Industrial Environment
Peter Kruse, Nelly Condorí-Fernández, Tanja E. J. Vos, Alessandra Bagnato, Etienne Brosse · 2013
[Context] Numerous combinatorial testing techniques are available for generating test cases. However, many of them are never used in practice. [Objective] Considering that learn ability plays a vital role in initial adoption or rejection of a technology, in this paper we aim to investigate the learnability of a combinatorial testing tool in an industrial environment. [Method] A case study research method was designed and conducted, by including i) the definition of learnability measures for test cases models built using a combinatorial testing tool. ii) A training program was also implemented. iii) Qualitative and quantitative evaluation based on a three-level strategy was carried out (Reaction, Learning, and Performance). [Results] At the first level, the tool was perceived as easy to learn by the trainees (from a five-point ordinal scale). However, at the second level, during hands-on learning, it changed slightly: According to the working diaries, there were major difficulties. At third level, analyzing the learning curve of each trainee, we observe that semantic errors made per each subject were reduced slightly over the time.