On Strategies to Exploit Dependencies Between Singing Voice Alignment and Separation
Théo Nguyen, Yann Teytaut, Axel Röebel · 2024
Voice separation and lyrics alignment have become essential tools for studying the singing voice and singing performance. At first glance, these tasks have no evident relations: the first estimates target vocals from a mixture of several sources, while the other determines the temporal occurrence of a text along a given audio. However, recent approaches, mostly based on deep learning, revealed potential helpful dependencies between both problems. The present study precisely focuses on informing each task with the other and highlights positive impacts on separation and alignment metrics. First, this paper covers the beneficial aspect of text alignment on singing voice separation using conditioning mechanisms. Second, new phonetic annotations of the MUSDB18 dataset are proposed and shared to improve text-informed singing voice separation. Finally, voice separation is used as a key pre-processing to reinforce a Connectionist Temporal Classification (CTC)-based lyrics aligner;