Body and Space : Combining Modalities for Musical Expression
Marco Donnarumma, Baptiste Caramiaux, Atau Tanaka · Goldsmiths (University of London) · 2013
This paper presents work in progress on applying a Multimodal interaction (MMI) approach to studying interactive music performance.We report on a study where an existing musical work was used to provide a gesture vocabulary.The biophysical sensing already used in the work was used as input modality, and augmented with several other input sensing modalities not in the original piece.The bioacoustics-based sensor, accelerometer sensors, and full-body motion capture system generated data recorded into a multimodal database.We plotted the data from the different modalities and offer observations based on visual analysis of the collected data.Our preliminary results show that there is complementarity of different forms in the information.We noted three types of complementarity: synchronicity, coupling, and correlation.