Making usability recommendations useful and usable
Rolf Molich, Robin Jeffries, Joseph S. Dumas · 2007
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Copyright 2006, ACM. Vol. 2, Issue 4, August 2007, pp. 162-179 This paper evaluates the quality of recommendations for improving a user interface resulting from a usability evaluation. The study compares usability comments written by different authors, but describing similar usability issues. The usability comments were provided by 17 professional teams who independently evaluated the usability of the website for the Hotel Pennsylvania in New York. The study finds that only 14 of the 84 studied comments (17%) addressing six usability problems contained recommendations that were both useful and usable. Fourteen recommendations were not useful at all. Sixteen recommendations were not usable at all. Quality problems include recommendations that are vague or not actionable, and ones that may not improve the overall usability of the application. The paper suggests characteristics for “useful and usable recommendations, ” that is, recommendations for solving usability problems that lead to changes that efficiently improve the usability of a product. Keywords Usability recommendation, usability comment, usability evaluation quality, usability evaluation problems, usability test report, usability evaluation report, usability testing, expert review, web evaluation, usability politics, persuasive communications.