Comparing affective and sensorial expressions in music performance by machine learning techniques
Luca Mion, Giovanni De Poli · Padua Research Archive (University of Padova) · 2007
We present a paradigm for expression understanding in music performance based on a joint semantic space, described by both affective and sensorial adjectives. Machine learning techniques were used to select the most relevant audio features allowing to recognize different expressive intentions both in the affective (Valence-Arousal) and sensory (Kinematics-Energy) spaces. By Sequential Forward Selection and Principal Component Analysis we derived a set of features for a general description. Good expression recognition was achieved using these cues. The positions of emotional and sensory adjectives in the feature spaces are in good agreement with their positions in the corresponding affective and sensorial spaces. Then, all the performances described by the selected features were projected on a joint 2D space. Thus we exploited the significance of the selected features in order to see how all the adjectives are jointly organized in the feature space, and to explain the found relations relating the space to a semantic interpretation and possible association among sensorial and affective labels. Three main cluster emerged: (a) Hard/Heavy/Angry, (b) Sad/Calm/Soft/, (c) Light/Happy. An interpretation of the clusters based on action and physical analogy can be devised. When using the physical analogy, force is often subjectively considered as the cause and movement (velocity or position) as the effect. The cause-effect relation is represented by the admittance Y which mathematically describes the dynamic mapping and the qualitative behaviour from force to velocity by an integral-differential equation. We can distinguish three kinds of basic behaviours: friction, which determines the amount of effect and thus scales the energy; elasticity, which opposes changes in force; inertia, which opposes changes in movement. This allows us the associate an interpretation to clusters (a) as related to the concept of energy and (low) friction, cluster (b) to elasticity, and cluster (c) to inertia.