An exploratory empirical eye-tracker study of visualization techniques for coverage of combinatorial interaction testing in software product lines
Kambiz Nezami Balouchi, Julien Mercier, Roberto E. Lopez-Herrejon · Journal of Systems and Software · 2024
Software Product Lines (SPLs) typically provide a large number of configurations to cater to a set of diverse requirements of specific markets. This large number of configurations renders unfeasible to test them all individually. Instead, Combinatorial Interaction Testing (CIT) computes a representative sample according to criteria of the interactions of features in the configurations. We performed an empirical study using eye-tracker technologies to analyze the effectiveness of two basic visualization techniques at conveying test coverage information of ten case studies of varying complexity. Our evaluation considered response accuracy, time-on-task, metacognitive monitoring, and visual attention. The study revealed clear advantages of a visualization technique over the other in three evaluation aspects, with a reverse effect depending on the strength of the coverage and distinct areas of visual attention. • Study of visualization techniques for interaction testing of Software Product Lines. • Analysis of visual attention using eye-trackers for coverage testing tasks. • Scatter plots and Parallel dimensions plots offer different performance trade-offs.