It’s not always what the eye can see – challenges in the evaluation of augmented reality
Kurtis Goad, Kevin C. Mangold, Susanne Furman · 2023
Augmented reality (AR) technology is developing at a fast pace. Usability evaluation methodologies for AR need to be updated to accommodate the increasing complexity of AR technology. Eye tracking metrics, which have been historically successful in traditional, often computer-based technology, have been underutilized when evaluating AR devices. Literature review shows documentation of usability methodology used to evaluate AR since 2000. But these reviews did not address the reasons why evaluators have chosen or excluded certain methods, like eye tracking. They also did not address certain evaluations, such as those performed in the industry sector, which are not typically published. This exploratory study addressed these gaps by deploying semi-structured interviews to gain an understanding of the experiences professionals have when evaluating AR technology. Our results show that participants rely most heavily on user feedback and questionnaires/surveys during evaluation. Most participants did not use eye tracking methods. They cited a number of challenges evaluating AR technology and the use of eye tracking, including difficulties in data collection and a lack of consistency and standards across devices.