Correlations of Evaluation Metrics for Voice Conversion: An Experimental Analysis
Arkapravo Nandi, Subhayu Ghosh, Md. Tousin Akhter, Sandipan Dhar, Nanda Dulal Jana · 2024
Voice conversion (VC) refers to the transformation of vocal features from one speaker to another speaker while keeping linguistic content unaltered. Recently, VC has seen rapid advancement with the development of deep learning techniques, becoming increasingly critical in applications such as voice assistive technologies, voice dubbing for movies, and privacy protection. A core challenge in VC research lies in the comprehensive evaluation of converted voice quality, speaker similarity, and intelligibility. This study presents an extensive experimental analysis aimed at investigating the correlations between objective and subjective evaluation metrics used in VC. A series of experiments using state-of-the-art (SOTA) VC models has been performed on several benchmark datasets. Objective metrics such as mel-cepstral distortion (MCD), modulation spectra distance (MSD), etc. are calculated, alongside subjective assessments including mean opinion score (MOS) for quality and similarity. This analysis reveals significant insights into the strengths and limitations of commonly used metrics, highlighting cases where different measures correlate well with each other and instances where they diverge. Our findings contribute to a better understanding of metric effectiveness in VC evaluation, offering guidelines for future research and development in the field.