A penny for your thoughts: Investigating the validity and reliability of think-aloud protocols for usability testing
Maaike J. van den Haak · 2008
Think-aloud protocols are a dominant method in usability testing.There is, however, only little empirical evidence on the actual validity of the method.This paper describes an experiment that compares concurrent and retrospective think-aloud protocols for a usability test of an online library catalogue.There were three points of comparison: usability problems detected, overall task performance, and participant experiences.Results show that concurrent and retrospective think-aloud protocols reveal comparable sets of usability problems, but that these problems come to light in different ways.In retrospective think-aloud protocols, more problems were detected by means of verbalization, while in concurrent think-aloud protocols, more problems were detected by means of observation.Moreover, in the concurrent thinkaloud protocols, the requirement to think aloud while working had a negative effect on the task performance.This raises questions about the reactivity of concurrent think-aloud protocols, especially in the case of high task complexity.