Dynamic Difficulty Adjustment in Audio Augmented Reality Games
Fotis Giariskanis, Yannis Kritikos, Eftychia Protopapadaki, Anthi Papanastasiou, Eleni Papadopoulou, Katerina Mania · Journal on Computing and Cultural Heritage · 2025
Incorporating Dynamic Difficulty Adjustment (DDA) and Conversational UIs (CUIs) in relatively unexplored head-worn Audio Augmented Reality (AAR), puts forward a combination of intelligent interaction modalities including hand and gaze interaction as well as spatial sound and voice input. Seamless interaction modalities driving a culturally engaging experience in AAR is a challenge. This article presents the design, implementation, and evaluation of an innovative, flow-aspired AAR experience for head-worn AAR, presented as a gamified historical narrative. User-driven DDA, spatialized auditory feedback, and gaze-based interaction are integrated with location-specific musical elements. The DDA component uses heuristic distributions based on round completion time, hints’ usage, remaining time, and additional time needed, weighted to calculate an overall score. Players regardless of their skill level enjoy the game experience based on customized difficulty. Upon completing AAR missions, the user composes their own personalized soundscape. The proposed AAR experience overcomes interaction challenges of personalized AAR, putting forward a voice-assisted system for universal access, also employing accurate hand and gaze interaction, while the user is moving in a busy museum environment. A thorough evaluation showed that combining voice, hand, and gaze interaction was of higher usability compared to voice-only and gaze-only interaction. The DDA component matched players’ individual styles, retaining cultural engagement.