Linguistic Extension of UTAU Singing Voice Synthesis and Its Application from Japanese to Mandarin
Hung-Che Shen · 2022
The Mandarin singing voice synthesis by UTAU is investigated. The UTAU software is a Japanese singing voice synthesis system that is widely used to create variable musical content. As a state-of-the-art singing voice synthesis system, UTAU synthesizes singing voices of reasonable quality. However, many efforts are still required to obtain multilingual singing synthesis. Japanese pitch changes only between syllables, while Mandarin pitch changes within the syllables. In addition, Mandarin needs pinyin to look up words in the dictionary. To use UTAU for Mandarin, we propose a linguistic extension of UTAU. The extension contains modules in the front and back ends. The front-end GUI module makes the lyrics more readable. The back-end pitch bend editor adds four tones of Mandarin to a melody. The application of UTAU software especially for Mandarin singing is presented and evaluated in subjective experiments. Then, Japanese and Mandarin singing voice syntheses are compared. As a result, the tonal production accuracy and naturalness of synthesized singing in Mandarin is significantly improved. Therefore, the extension is to offer a direct way for other languages.