Real-time Visualization of Musical Expression.
Roberto Bresin · 2004
A system for real-time feedback of expressive music performance is presented. The feedback is provided by using a graphical interface where acoustic cues are presented in an intuitive fashion. The graphical interface presents on the computer screen a three-dimensional object with continuously changing shape, size, position, and colour. Some of the acoustic cues were associated with the shape of the object, others with its position. For instance, articulation was associated with shape, staccato corresponded to an angular shape and legato to a rounded shape. The emotional expression resulting from the combination of cues was mapped in terms of the colour of the object (e.g., sadness/blue). To determine which colours were most suitable for respective emotion, a test was run. Subjects rated how well each of 8 colours corresponds to each of 12 music performances expressing different emotions.