Editorial: The importance of empirical work for software engineering papers
Lee J. White · Software Testing Verification and Reliability · 2002
The importance of empirical work for software engineering papers I have noticed that in recent years there has been a growing insistence that software engineering papers, both in journals and conference proceedings, should include empirical results.I can recall a time when a paper that had new and interesting ideas, submitted to either a journal or a conference, could be deemed worthy of publication on that basis alone.Today, reviewers and editors expect empirical work to be included in papers submitted for publication.This is not only a requisite for research papers, but also for industrial papers as well, although this empirical aspect may well be in the form of case studies.The reason for this is the need for validation of assumptions or other claims, and the verification of results in the paper.In software engineering, as opposed to some other fields in computer science and engineering, it is often not easy to verify your results analytically, and so an empirical study may be the only way to provide such verification.Reviewers do not always insist on a fully-fledged experiment, but do require at least a lightweight empirical study, still carefully thought out and executed.Also, most reviewers will no longer simply accept the results of empirical study or full experiment, but insist upon some sort of statistical analysis to demonstrate statistical significance of the results.I have been somewhat surprised by the necessity for statistical significance, but experimental results that appear compelling often fail to be statistically significant; you can't just eyeball the data and be confident that it couldn't come out that way by coincidence.I tend to be fairly theoretical in my general approach to research, and so years ago I was not trained well in experimental approaches; I was intimidated by the need for experiments and statistical analysis.Now I require that all my students provide some empirical study in their MSc or PhD research, and often insist that full experiments be designed and executed.Recently a friend, Lionel Briand, who is a respected researcher in experimental software engineering, congratulated me on an experimental study that a PhD student of mine had completed, and that I had presented.I was very proud, considering the source of this praise.So I can now claim that any of you can master these experimental and statistical techniques in presenting your ideas, arguing the case for their usefulness and adoption!The three papers in this issue clearly illustrate several approaches to empirical work.Two of the papers provide fully-fledged experiments, carefully designed with thorough analysis: 'Can