Generating music from poems using neural networks
Horia-Ioan Iancu, Ștefan Trăușan-Matu · 2021
Sonification is the process of using sound to convey or add a new dimension to information.In particular, textual humancomputer interaction is enhanced if sound is added.In order to sonify a poem, a quantified representation of emotions in the form of sentiment dictionaries would be necessary, as well as knowledge of music theory.Diverging from previous research, a method that uses deep neural networks to combat both of these problems is presented.In order to establish an emotional connection between text and music, the general sentiment of a poem is first extracted.An analysis of the poem's musicality, expressed in the form of rhythm, rhyme and punctuation, is then used as the basis for creating a simple rule-based melody.Using this melody as conditioning, a piano accompaniment is then produced using a neural network.Results show that the accompaniments generated adequately reflect the mood of a poem, while also being pleasant to listen to.