Extended Reality Educational System with Virtual Teacher Interaction for Enhanced Learning
Fotis Liarokapis, Václav Milata, Filip Škola · Multimodal Technologies and Interaction · 2024
Advancements in technology that can reshape educational paradigms, with Extended Reality (XR) have a pivotal role. This paper introduces an interactive XR intelligent assistant featuring a virtual teacher that interacts dynamically with PowerPoint presentations using OpenAI’s ChatGPT API. The system incorporates Azure Cognitive Services for multilingual speech-to-text and text-to-speech capabilities, custom lip-syncing solutions, eye gaze, head rotation and gestures. Additionally, panoramic images can be used as a sky box giving the illusion that the AI assistant is located at another location. Findings from three pilots indicate that the proposed technology has a lot of potential to be used as an additional tool for enhancing the learning process. However, special care must be taken into privacy and ethical issues.