Exploring Interaction Modalities for Hands-Free Menu Navigation: Insights from Immersive Experimentation
Israel Gomez, Samantha L. DeNicola, Neil G. McHenry, Gregory E Chamitoff, Ann McNamara · 2024
In response to the growing need for streamlined hands-free interfaces across diverse fields such as space exploration, education, and industry, we performed a study to determine the most effective method of interacting with Graphical User Interface (GUI) menus in an augmented reality (AR) setting. Focusing on astronauts as the target users, we explored different interaction modalities within an immersive Virtual Reality (VR) environment. Through a suite of user studies, we compared five interaction modalities: voice commands, gaze tracking, eye tracking, popup buttons, and gestures. The goal of this work was to identify the most effective and user-friendly approach to optimize user efficiency and efficacy in hands-free tasks. Participants performed a standardized menu navigation task using each interaction modality. The experiments were designed to simulate scenarios where manual dexterity is restricted, such as during extravehicular activities in space or for workers who require hands-free operation in hazardous environments. Our findings reveal valuable insights into the usability and efficiency of each interaction modality. Gaze, eye tracking, and voice commands demonstrated comparable performance, although their effectiveness varied depending on the individual’s eye coordination and calibration. Gestures, while intuitive, required additional refinement to ensure reliable recognition and to minimize the risk of accidental triggers. Pop-up buttons emerged as the preferred modality among participants, boasting the shortest task completion time. The implications of this work extend beyond the astronaut context. Users in various industries can benefit from effective hands-free menu navigation in both AR and VR settings. The results of this study will inform the design and implementation of future interaction systems, enabling improved efficiency and comfort for individuals operating in constrained environments.