Threats to validity in controlled experiments in software engineering
Eudis Teixeira, Liliane Sheyla da Silva Fonseca, Sérgio Soares · 2018
Context: Every experimental study has some threats to validity hindering its results. Goal: Improve software engineering controlled experiments quality by better understanding threats to validity control process. Method: A systematic Survey was executed to collect information from software engineering controlled experiments specialists. Data was quantitative and qualitatively analyzed. Results: 115 researchers took part in the study. Most of them (78.26%) consider extremely important to identify threats to validity during experiments planning to adjust it reducing the probability of threats to validity impacting experiment execution. Conclusions: Results bring participants point of view about identifying controlled experiments threats to validity. However, the study reveals some concerns since a considerable number (18.26%) of participants are not aware of threats to their studies or to new threats raised by actions took to address previous threats, hindering results validity.