Production of Mixed Reality Interactive Content using Motion Capture based Virtual Avatar
Hyeonyong Yang, Sumin Kong, Jiwon Lee, Jinmo Kim · Journal of the Korea Computer Graphics Society · 2025
This study produces mixed reality (MR) interactive content that mixes motion capture-based virtual avatars in real space and interacts in real time. First, an integrated development environment is established for producing interactive content using the OptiTrack motion capture system and Meta Quest 3 MR HMD (Head Mounted Display) in the Unity 3D engine. Then, voice and action synchronization functions are implemented in a way that a virtual avatar expressed by full-body tracking of an actor wearing a suit with markers attached through optical motion capture and a MR user communicates and interacts in real time. Based on this, an interactive content about medication assistance is produced for the purpose of practical application of an experiential environment where a virtual avatar and a MR user communicate through conversation and share emotions and situations. Finally, the user experience of the produced content and the MR environment composed of motion capture-based virtual avatars are analyzed through survey experiments to compare the effects on immersion and satisfaction compared to GUI-based interactions with a fixed pattern. As a result, it was confirmed that a MR experience environment composed of a virtual avatar that can flexibly cope with a given situation and communicate naturally through an actor can provide an interesting experience to users and allow them to immerse themselves more in the purpose of the content and increase overall satisfaction with the content.