Immersive creativity: a framework for expressive and creative interaction in virtual and mixed reality inspired by musical thinking
César Rennó-Costa, Alyson Matheus de Carvalho Souza, Stefane de Assis Orichuela, João Anisio Marinho Da Nobrega, Jônatas Manzolli, Cleber da Silveira Campos · Frontiers in Virtual Reality · 2026
This paper proposes a novel framework for designing immersive creative systems in Virtual and Mixed Reality (VR/MR) by drawing upon the rich, centuries-old domain of music. It argues that the principles underlying musical creation provide a powerful blueprint for interaction design. By analyzing the cognitive processes of composition and performance, this work synthesizes a design framework organized around four key pillars: (1) the Visualization of the Sonic Space, which translates abstract musical ideas into navigable environments; (2) the design of Embodied Instruments as gestural interfaces; (3) the implementation of Structured Freedom through guiding constraints; and (4) a paradigm shift towards The Composer as World-Builder, creating environments for interpretation. The paper then demonstrates the framework’s practical application by outlining its use in designing real-world immersive experiences. The goal is to guide development away from simple virtual tools and towards interpretive, collaborative environments that amplify human creative potential.