The dreaded desk reject
Robert M. Hierons · Software Testing Verification and Reliability · 2015
After months of carefully developing a paper, one submits it to a journal and 2 weeks later it is rejected without review (a ‘desk reject’). Most of us have experienced this scenario; I certainly have. What has happened, and why? Unfortunately, the ‘desk reject’ is part of the journal editor's job, and I suspect most do not enjoy it. Papers are desk rejected for three main reasons. The first is simply that the paper does not fit with the journal's scope. The second is that the paper appears to have deficiencies that mean that it is not worth sending out to review. This might be a result of weaknesses in the research or in the presentation (usually the standard of English). With both reasons, all benefit from the decision: reviewer time is saved, and authors receive faster feedback. The final reason is plagiarism, which was the topic of a previous editorial. Authors can do certain things to make a desk reject less likely. Does the research fit with the journal's scope and, if so, is this clear from the paper? Is the standard of English acceptable? Is the work of a similar standard to, or better than, other papers in the journal? I find that one of the best ways of determining the latter is simply by reading many relevant papers from the journal – and if there are not relevant papers then this might say something about scope! Naturally, there is also the need to assess the overall contribution of the research: is the work novel and potentially significant; is it developed rigorously; and is there a good evaluation. Finally, we can all benefit from discussing these things with our peers – and maybe then the desk reject will become a thing of the past. This issue contains three papers that are related to two important topics that are often considered by quite different communities: regression testing and fault localisation. The first paper concerns regression testing, and the second paper combines the two themes: the authors generalize a fault localisation method (delta debugging) and evaluate this in the context of regression testing. The third paper explores fault localisation for JavaScript. Ghaith et al.explore the regression testing process when carrying out performance testing. Such regression testing aims to find performance anomalies in which there is a significant change in time taken or resource utilisation. This paper is motivated by two practical issues. First, there is a need for automation. Second, the time taken by a transaction depends on the system workload, such as what other transactions are taking place, and there is a need to factor out any changes in workload. The authors propose an automated technique that is based on a queuing network model and is independent of the workload. In this approach, the detection of performance anomalies is based on a metric called the transaction profile: the time taken by a transaction if no other transactions are being carried out. The value of transaction profile is estimated on the basis of the queuing network model and the observed transaction response time and resource utilisation. The work was evaluated through an industrial case study (Recommended by A. Pretschner). Delta debugging is a well-known technique for taking a failing-test case and producing a simplified (failing) test case, the motivation being that this should simplify the debugging process. Groce et al.generalize delta debugging to cause reduction in which a test case is simplified while retaining a property of interest such as coverage. The authors note that standard delta debugging algorithms can be adapted for use in cause reduction. The approach is evaluated in the context of regression testing, where the aim is to reduce the execution time of a regression test suite while preserving code coverage. The results of an empirical study are promising, with the reduced test suites being significantly more efficient while having similar effectiveness. (Recommended by G. Fraser and D. Marinov). Ocariza et al. look at automated fault localisation for JavaScript. The paper focuses on DOM-related faults, pointing to evidence that these are relatively common and arguing that they are also particularly difficult to fix. They develop an approach that uses dynamic backward slicing and have implemented their technique in a tool (AUTOFLOX). They also applied AUTOFLOX to a number of case studies, with it being able to locate 96% of the seeded faults and 20 real faults in real-world applications. (Recommended by L. Zhang).