Musical Content Processing for Expressive Gesture Applications in Interactive Multimedia
Marc Leman, Antonio Camurri · 2004
Background in musicology: Defining the expressive nature of musical content (and meaning) has been a major preoccupation of many researchers in systematic musicology, aesthetics, phenomenology, hermeneutics and semiotics. Many theories of musical expressive meaning formation have been formulated but few have been connected with the physical signals that carry the information. With a lack of evidence-based grounding a theory remains a metaphor, and has no instrumental impact on reality. Modern cognitive/systematic musicology, however, is grounded on evidence-based research and computational processing, and works towards an elaborate testing environment for the development of theories that, ultimately, aim at closing the old gap between sound and musical practice. Background in computing: Interactive multimedia applications have a main focus on the processing of information streams but they often suffer from a serious lack of processing capabilities that cope with the nature of human information processing. If these systems have to interact in an intelligent and spontaneous way with humans, then it is evident that their communication capabilities should deal with human cognitive, affective/emotive, and motoric faculties. There are indeed many occasions where users may want to interact with machines at levels that go beyond verbal descriptions, using indications of perceived qualities, or making expressive movements. Progress in the domain of non-verbal gesture-based content processing, therefore, is urgently needed and the artistic musical context offers