An Environment for Gestural Interaction with 3D Virtual Musical Instruments as an Educational Tool
Christos Garoufis, Athanasia Zlatintsi, Kosmas Kritsis, Panagiotis Paraskevas Filntisis, Vassilis Katsouros, Petros A. Maragos · 2019
This paper presents a finalized version of an environment intended for performance and gestural interaction with three-dimensional virtual musical instruments, developed as a part of a larger educational platform, the iMuSciCA workbench. The environment can employ either a Leap Motion or a Kinect sensor, and enables interaction with a variety of virtual musical instruments, namely virtual interpretations of a bichord, a xylophone, a drumming set, a guitar and an upright bass, by means of performing and recognizing hand gestures similar to the ones needed to play their physical counterparts. In order to showcase the usability of the platform in an educational context and measure its effectiveness, we designed a scenario, where the user tries to keep a steady rhythm while drumming. A usability study of the above scenario, involving 22 users, demonstrates that the audiovisual feedback can actually provide assistance to the user.