Evaluating Severity Rating Scales for Heuristic Evaluation
Sascha Herr, Nina E. Baumgartner, Tom Gross · 2016
The heuristic evaluation is a widely applied discount usability evaluation method. Experts use the method to identify usability issues in interfaces and to rate their severity in order to establish a prioritization of resource allocation. However, in practice, there are often large discrepancies between the individual severity ratings of experts, indicating challenges with the rating process and doubtful accuracy of ratings. This paper discusses these challenges by drawing from research on psychometrics, proposes solutions and reports preliminary findings of an empirical online study.