A Real-Time Music Emotion Modulation System for Soundtracks
Bing Yen Chang, Andrew Horner · University of Michigan Library Repository · 2017
Many entertainment media contain thematic soundtracks, oftentimes achieved by repeatedly reprising a core music piece via modulation of musical parameters, such as Tempo, Dynamics, Pitch, and Mode, to fit the emotional characteristics of the varying scenes. Moreover, interactive media soundtracks need to be responsive to user interaction. This paper implements a real-time MIDI system that automates this emotion modulation process. The system takes as input Valence-Arousal levels to perform real-time musical parameter modulation of the user-specified music piece. Arousal is probabilistically mapped to a weighted sum of Tempo and Dynamics. Valence is mapped with Positive as Major and Negative as Minor, utilizing Relative, Natural, and Harmonic Minor, together with the Pitch register for further shading. Each input event also triggers a MIDI instrumentation change to enhance user engagement. Modulation of Canon in D towards the Happy, Calm, Sad, and Angry quadrants evoked joyous, meditative, depressive, and vengeful emotional characteristics respectively. Real-time modulation, particularly with a fast stream of quadrant-changing inputs, provides a varied and interesting musical experience akin to an emotionally moody music box. This system provides a framework for audio engineers and soundtrack designers to automate production of thematic and adaptive soundtracks for films, video games, and virtual reality. © 2017 Bing Yen Chang et al.