Plat Forms: Contests as an Alternative Approach to SE Empirical Studies in Industry
Lutz Prechelt, Ulrich Stärk, Freie Universität · 2013
Abstract—Plat Forms is a set of three studies that search for process and product properties emerging from the use of specific web development platforms. Its first key property is the use of a contest format; this leads to easier, broader, and more uniform data collection compared to on-site industrial studies. As a second key property, the three instances do not merely serve to extend or corroborate each others ’ results, they also serve as a time series for characterizing changes in the underlying software development reality over the course of a few years – an important aspect that has received little attention in empirical software engineering research so far. I. PLAT FORMS Plat Forms is a set of three quasi-experiments 1. Its goal is to compare industrial development technologies (tools, frameworks, components) using realistic teams and development processes, but factoring out the complexity effects of industrial-sized tasks and workplace effects. The key idea of the Plat Forms research approach is to represent the study to the outside world as a contest. A. Description The three Plat Forms studies were carried out in 2007, 2011, and 2012. The external form of each was that of a contest. The abstract of the 2007 announcement [1] read as follows: “Plat Forms ” is a competition in which topclass teams of three programmers compete to implement the same requirements for a web-based system within 30 hours, each team using a different technology platform (Java EE,.NET, PHP, Perl, Python, or Ruby on Rails). The results will provide new insights into the real (rather than purported) pros, cons, and emergent properties of each platform. The evaluation will analyze many aspects of each solution, both external (usability, functionality, reliability, performance, etc.) and internal (structure, understandability, flexibility, etc.). We asked for (and largely got) jelled teams of professionals to apply for participation with the technology they mastered best. The teams would work as volunteers; we had promised only fun and fame, not fortunes 2. 9 teams participated in 2007