An Analysis of the Empirical Software Engineering over the last 10 Editions of Brazilian Software Engineering Symposium
Davi Emanuel Silva Monteiro, Rômulo Gadelha, Thayse Alencar, Bruno Neves, Italo Yeltsin, Thiago Queiroz Gomes, Mariela I. Cortés · 2017
Empirical evaluations developed in the software engineering area have been widely applied as a formalism to validate and ensure the credibility of the works proposed by the researchers. Even though the adoption of empirical evaluation techniques has gained popularity in recent years, its application has been questioned both qualitatively and quantitatively. This study aims at analyzing how empirical software engineering research has evolved in the Brazilian Symposium on Software Engineering (SBES) community. We performed a controlled quasi-experiment, using published papers over the last 10 years in SBES. Our experiment was divided into two phases: classification by type and quality assessment of the main empirical types. In the first phase, the sample was 201 papers; in the second one, the sample decreased to 126 papers. The results have shown failures and gaps in the application of empirical methods when assessing the quality of the Software Engineering works. We believe that we can contribute to improve how the studies were conducted and consequently help to produce more reliable results, reducing or eliminating biases: an important qualitative factor in scientific work. In addition, due to the lack of assessment supporting tools, we developed a theoretical protocol to support the assessment process and proposed improvements for papers that obtained below-expected rates.