SISA: Engaging with Performing Arts Through Spatialized Interaction with Segmented-audios in Immersive Environments
Sirui Wang, Yuqi Wang, Michelle Lui, Ray Lc · 2025
Intangible cultural heritage (ICH) performing arts, particularly their auditory elements, face significant challenges in transmission and engagement among younger generations due to their inherently temporal and passive nature of appreciation. To address this challenge, we present a Spatial Interaction-based Segmented-Audio (SISA) system in Virtual Reality (VR) that transforms temporal auditory experiences into interactive spatial explorations. Our approach segments audio content and applies t-SNE algorithm for spatial clustering within VR environments. In this paper, we demonstrate SISA implementation through four VR scenes featuring two ICH genres - Peking Opera and Meshrep - each incorporating 5-second and 10-second audio segments. Through user testing with 16 participants, we explored users’ perceptions and interactions with SISA system within these VR environments. Our work advances technology-mediated cultural preservation by establishing a system and framework for converting temporal performances into navigable spatial experiences, creating new pathways for ICH engagement.