Enhancing Immersive Learning: An Exploratory Pilot Study on Large Language Model‐Powered Guidance in Virtual Reality Labs
Amir Abbas Yahyaeian, Morteza Sabet, Jing Zhang, Alan Jones · Computer Applications in Engineering Education · 2025
ABSTRACT Laboratory learning is central to engineering education, yet physical lab access is often limited by resource constraints, safety requirements, and instructor availability. Immersive virtual reality (IVR) environments can expand access, but learners working independently may lack the procedural guidance required to progress confidently. This study presents a pilot implementation of an intelligent virtual instructor (IVI) for a VR mechanical fatigue testing laboratory, enabling real‐time, context‐aware guidance without live instructor supervision. Using a design‐based approach, student interactions ( n = 6) were observed to identify common points of difficulty and document instructor scaffolding strategies. These observations were used to construct a state‐conditioned dataset linking learner location and task progression to appropriate guidance responses, and a text‐generation model was fine‐tuned to produce instructional prompts based on this state. The fine‐tuned model generated guidance that was procedurally accurate, contextually relevant, and clearly articulated, indicating that an IVI can effectively support task execution in IVR laboratories. This work establishes a proof of concept for state‐grounded virtual instruction and provides a foundation for evaluating skill transfer and learning outcomes in future large‐scale studies.