Video Conferencing With Predictive Generation and Collaborative Computation Across Mobile Headsets
Yili Jin, Xize Duan, Kaiyuan Hu, Fangxin Wang, Xue Liu, Jiangchuan Liu · IEEE Transactions on Mobile Computing · 2025
Virtual Reality (VR) has emerged as a transformative platform for remote collaboration, but its adoption for video conferencing is hindered by challenges related to facial expression reconstruction and computational resource constraints, especially on economical mobile VR headsets. This paper introduces a novel system for VR video conferencing that addresses these challenges through two key modules: Predictive Generation and Collaborative Computation. Predictive Generation leverages multimodal inputs, including voice, head motion, and eye blinks, to synthesize realistic facial animations with low latency, eliminating the need for high-precision hardware. Collaborative Computation enhances computational efficiency by employing a game-theoretic framework for resource sharing among users. Experimental evaluations demonstrate that our system delivers immersive and realistic VR video conferencing experiences with superior facial expression reconstruction and efficient resource utilization. Our approach makes VR video conferencing more accessible and practical for a broader audience across mobile headsets.