An evaluation of voice conversion with neural network spectral mapping models and WaveNet vocoder
Patrick Lumban Tobing, Yi-Chiao Wu, Tomoki Hayashi, Kazuhiro Kobayashi, Tomoki Toda · APSIPA Transactions on Signal and Information Processing · 2020
An evaluation of voice conversion with neural network spectral mapping models and WaveNet vocoder patrick lumban tobing, 1 yi-chiao wu, 1 tomoki hayashi, 1 kazuhiro kobayashi 2 and tomoki toda 2 This paper presents an evaluation of parallel voice conversion (VC) with neural network (NN)-based statistical models for spectral mapping and waveform generation. The NN-based architectures for spectral mapping include deep NN (DNN), deep mixture density network (DMDN), and recurrent NN (RNN) models. WaveNet (WN) vocoder is employed as a high-quality NN-based waveform generation.In VC, though, owing to the oversmoothed characteristics of estimated speech parameters, quality degradation still occurs.To address this problem, we utilize post-conversion for the converted features based on direct waveform modifferential and global variance postfilter.To preserve the consistency with the post-conversion, we further propose a spectrum differential loss for the spectral modeling.The experimental results demonstrate that: (1) the RNN-based spectral modeling achieves higher accuracy with a faster convergence rate and better generalization compared to the DNN-/DMDNbased models; (2) the RNN-based spectral modeling is also capable of producing less oversmoothed spectral trajectory; (3) the use of proposed spectrum differential loss improves the performance in the same-gender conversions; and (4) the proposed post-conversion on converted features for the WN vocoder in VC yields the best performance in both naturalness and speaker similarity compared to the conventional use of WN vocoder.