Exploring Interaction in a Virtual Music Studio through Gesture Recognition on Smartwatches and HMD Devices
Murilo Santos de Castro, Pedro Raphael Inácio Gomes, Thamer Horbylon Nascimento · 2023
This work proposes the development of an innovative virtual environment that simulates a music studio, exploring interaction through gesture recognition on smartwatches and mobile virtual reality devices (mVR). By utilizing Multiple Dimensional Dynamic Time Warping (MDDTW) for gesture recognition, users will be able to interact intuitively and immersively with virtual musical instruments. The smartwatch acts as an input device, capturing user gestures and transmitting them to the virtual environment. Moreover, the interaction also allows for the modification of musical samples and the visualization of instrument details. The combination of smartwatches and mVR devices offers a unique experience, expanding possibilities for expression and control for users. It is expected that this approach will drive human-computer interaction in various domains, showcasing new paths for the integrated use of established technologies in the market. The study aims to provide an engaging and immersive experience in the musical context, catering to both music enthusiasts and professionals, while contributing to the ongoing evolution of technological interaction.