Investigating the cross-modal relationship between music and motion in an improvised music production context
Shen Li · White Rose eTheses Online (University of Leeds, The University of Sheffield, University of York) · 2016
When responding to music, humans move their bodies with various motional patterns varying in speed, spatial dimensions and continuity. This cross-modal association has been widely examined by motion-induction studies; however, it is less studied in a music-production context. This research examined the transformation from the visualized motion patterns into musical characteristics of performed sounds in a creative production environment. Pianists were required to play expressively on either a single tone or a sequence of musical tones several times after watching different video stimuli. The results revealed that perceived speed in visual stimuli had an impact on performance tempo; walking distance (increasing/decreasing) from the camera influenced performance volume; and movement continuity affected performance articulation. This is consistent with previous findings on music-motion analogies. This research also revealed several potential correspondence patterns: Visualized motional height had an impact on musical articulation (higher/staccato, lower/legato), and motional speed may influence musical loudness (faster/louder, slower/softer). This research implied that expressive performance intention of pianists is associated with specific movement patterns. Keywords: cross-modal mapping, motion-music association, embodied cognition, musical production, expressive performance.