An Empirical Comparison of Lab and Remote Usability Testing of Web Sites
Tom Tullis, Stan Fleischman, Michelle McNulty, Carrie Cianchette, Marguerite Bergel · 2002
This paper presents the results of two studies comparing traditional lab-based usability testing and remote Web-based usability testing of Web sites. Two sites were tested: an employee benefits site and a financial information site. The remote tests used an automated technique whereby the users participated from their normal work locations using their normal browser, and there was no real-time observa-tion. Tasks were presented to the user in a small browser window at the top of the screen that was also used to cap-ture their input. Results showed high correlations between lab and remote tests for the task completion data and the task time data. Subjective ratings of the sites did not corre-late as well. The most critical usability issues with the sites were identified by both techniques, although each tech-nique also uniquely uncovered other issues. In general, the results indicate that both the lab and remote tests capture very similar information about the usability of a site. Each type of test appears to offer its own advantages and disad-vantages in terms of the usability issues it can uncover.