Large multimodal models in virtual reality: advancing immersive experiences through artificial intelligence integration
Liang-Yin Kuo, Ng Pei Ying, Chen-Yu Chuang, Yao-Yun Hsiao, Jing‐Chung Shen, Chia-Fang Hsu · IET conference proceedings. · 2025
Recent advances in artificial intelligence, specifically in large multimodal models (LMMs), have spurred new possibilities in virtual reality (VR) environments. LMMs, capable of processing and integrating diverse forms of data such as text, audio, video, and sensory inputs, are poised to enhance VR systems by creating more intuitive, adaptive, and immersive experiences. This paper explores the synergy between LMMs and VR, outlining key applications, technical challenges, and future opportunities. We delve into how LMMs improve user interaction, expand content generation capabilities, and foster more realistic virtual environments.