Evaluating software engineering methods and tool—part 3
Barbara Ann Kitchenham · ACM SIGSOFT Software Engineering Notes · 1996
In the first article about the DESMET evaluation methodology, I identified nine different types of evaluation:1. Quantitative Experiments2. Quantitative Case Studies3. Quantitative Surveys4. Feature Analysis --- Screening mode5. Feature Analysis --- Case Study6. Feature Analysis --- Experiment7. Feature Analysis --- Survey8. Qualitative Effects Analysis9. Benchmarking.In the last article I discussed various technical criteria that would influence your choice of an evaluation method. In principle, you should select the simplest evaluation method appropriate for your specific evaluation requirements and organisational capability. However there are a number of constraints that can influence your final choice of evaluation method:• the elapsed time that is needed for the different evaluation options;• the confidence that a user can have in the results of an evaluation;• the cost of an evaluation;This information may be useful in two circumstances:1. When you want to decide between a quantitative case study and a quantitative formal experiment and there are no clear technical reasons for preferring one to the other.2. When you want to ensure that the evaluation you have decided to undertake is feasible.These issues are discussed in this article.