Glow-WaveGAN 2: High-quality Zero-shot Text-to-speech Synthesis and Any-to-any Voice Conversion
Yi Lei, Shan Yang, Jian Cong, Lei Xie, Dan Su · Interspeech 2022 · 2022
The zero-shot scenario for speech generation aims at synthesizing a novel unseen voice with only one utterance of the target speaker.Although the challenges of adapting new voices in zero-shot scenario exist in both stages -acoustic modeling and vocoder, previous works usually consider the problem from only one stage.In this paper, we extend our previous Glow-WaveGAN to Glow-WaveGAN 2, aiming to solve the problem from both stages for high-quality zero-shot text-to-speech and any-to-any voice conversion.We first build a universal Wave-GAN model for extracting latent distribution p(z) of speech and reconstructing waveform from it.Then a flow-based acoustic model only needs to learn the same p(z) from texts, which naturally avoids the mismatch between the acoustic model and the vocoder, resulting in high-quality generated speech without model fine-tuning.Based on a continuous speaker space and the reversible property of flows, the conditional distribution can be obtained for any speaker, and thus we can further conduct highquality zero-shot speech generation for new speakers.We particularly investigate two methods to construct the speaker space, namely pre-trained speaker encoder and jointly-trained speaker encoder.The superiority of Glow-WaveGAN 2 has been proved through TTS and VC experiments conducted on LibriTTS corpus and VTCK corpus.