Between immersion and usability: A comparative study of 2D and mixed reality interfaces for remote music making
Alberto Boem, Matteo Tomasetti, Luca Turchet · International Journal of Human-Computer Studies · 2025
Networked Music Performance (NMP) systems have traditionally focused on optimizing audio transmission for remote collaboration, relying on basic 2D video feeds that fail to provide the spatial awareness and visual cues crucial for coordinated musical performance. While Mixed Reality (MR) technology offers promising solutions for enhanced musical collaboration, its integration into NMP systems remains understudied, particularly regarding the balance between immersion and usability. This study investigates this gap by comparing traditional 2D video conferencing setups with MR environments using point cloud representations and spatial audio. Through an experimental study with professional electric guitar players (N=16) and a simulation-based approach with pre-recorded setup, we examined how both 2D video and MR setups affect musicians’ performance, movement, and sense of presence using quantitative movement analysis and qualitative feedback. Head movement analysis revealed significantly reduced motion in MR compared to 2D conditions (p < 0.001), while detrended fluctuation analysis showed more structured movement patterns in MR (p = 0.0028). Post-task evaluations demonstrated that while MR enhanced immersion and presence (p < 0.001), participants reported significantly better performance coherence and reduced technical interference with 2D videos (p < 0.001). Spatial audio features showed minimal impact on user experience, with visual elements dominating attention. The findings highlight a fundamental tension between immersion and physical expression in MR music performance, suggesting that future systems should prioritize ergonomic considerations alongside technological capabilities. This research contributes to understanding the potential and the limitations of MR technologies in musical collaboration within the emerging Musical Metaverse.