Real-Time Voice-Based LLM Integration for XR Tutoring: A Prototype Implementation
Ali Geriş, Günter Alce · 2026
This paper presents a lightweight architecture for AI-driven tutoring in extended reality (XR) environments, integrating OpenAI's GPT-4o real-time API directly into Unitybased virtual reality (VR) to enable seamless voice-native interaction without external speech modules. Designed for taskbased learning, the tutor provides immediate, context-sensitive verbal feedback with adaptive brevity and low latency. A blockbased programming task demonstrates real-time evaluation, where user-assembled logic structures are analyzed and corrected through spoken responses. Across ten structured sessions and forty voice interactions, the prototype achieved consistent sub500 ms latency, generated under four minutes of total speech, and averaged $0.005 per turn. These results demonstrate the practical feasibility of low-overhead, real-time AI tutoring in XR and provide a foundation for future multimodal, context-aware learning environments.