Development and evaluation of a fully expressive avatar control system for communication and collaboration in virtual reality
Yuanjie Wu · University of Canterbury Research Repository (University of Canterbury) · 2020
In current virtual reality (VR) systems, making avatars expressive in non- verbal behavior (body movement, hand gestures, facial expressions, and eye gaze) is difficult due to complex sensory integration and data fusion requirements. Hence, the social impacts of avatar expressiveness in terms of non-verbal behavior have not yet been thoroughly investigated in shared virtual environments. I presented a novel expressive avatar system and conducted user studies to evaluate the system. The goal of the expressive avatar system was to develop it using off-the-shelf technology that provides an accurate, contactless, and natural interaction experience. The system consists of three parts. A customized multiple depth-camera system (MS Kinect v2) was developed for full-body tracking regardless of the user’s orientation. A multiple short-range depth-camera (Leap Motion) system was developed to provide a controller- free experience for enlarged natural hand gesture interaction. An avatar integration system was developed for avatar control and rendering with customized facial expressions such as the mouth and eye movement. To integrate these three systems I have designed a highly expressive avatar control system framework that includes a novel adaptive weighting data fusion method, an enlarged usable hands fusion method, and a robust avatar control algorithm. The system was evaluated in two user studies. The first user study ex- plored the effects of a depth-sensor-based avatar system on social behavior and performance in the single-user simulated communication scenario. The second user study explored the impact of different levels of avatar expressive- ness on collaboration behavior through a virtual charade game. My results demonstrated that a highly expressive avatar control system increased virtual body ownership and agency, improved user experience, and produced better non-verbal performance in single-user simulated communication scenarios. Furthermore, users felt a deeper sense of social presence and more attraction when interacting with a user who was using a highly expressive avatar in collaborative tasks. In summary, this thesis provides novel technical contributions on how to develop highly expressive avatar systems for communication and collaboration in shared virtual environments and evidence of how highly expressive avatars can benefit users in these scenarios.