Think Aloud: Can Eye Tracking Add Value in Detecting Usability Problems?
Alekhya Paruchuri · RIT Scholar Works (Rochester Institute of Technology) · 2012
The protocols of Think Aloud and Eye Tracking, in their own unique way have proven to be great methods to understand users' thought processes, and their mental models when interacting with interfaces. However the effectiveness of the combination of the two protocols in discovering usability problems has not been explored. This study aimed to discover if the addition of Eye Tracking data (fixations and scan movements) to the traditional protocol of Think Aloud can uncover more usability problems. Web users were split into three groups: Eye Tracking Only (ET), Think Aloud Only (TA), and Eye Tracking and Think Aloud Only (ET+TA). Participants in all conditions were asked to complete two tasks on two websites each. Along with questionnaires, eye movement data was collected for conditions with the Eye Tracking aspect and verbalizations were collected for conditions with the Think Aloud aspect. The analysis of the data showed that the total number of usability problems (not unique) identified by the participants in the `Eye Tracking and Think Aloud' (ET + TA) condition was higher than the other two conditions. However, a Tukey HSD post-hoc test revealed that the differences between `ET + TA' and the `Eye Tracking Only' (ET) conditions was non-significant. The analysis also which resulted in non-significant differences between the conditions `Eye Tracking' (ET) and `Eye Tracking and Think Aloud' (ET + TA) led to inconclusive results on whether the Think Aloud method is disruptive or not. This may lead future researchers to develop robust practice sessions to help participants verbalize and create evaluation rules for eye movement data.