MTI-Net: A Multi-Target Speech Intelligibility Prediction Model
Ryandhimas Edo Zezario, Szu‐Wei Fu, Fei Chen, Chiou‐Shann Fuh, Hsin‐Min Wang, Yu Tsao · Interspeech 2022 · 2022
Recently, deep learning (DL)-based non-intrusive speech assessment models have attracted great attention.Many studies report that these DL-based models yield satisfactory assessment performance and good flexibility, but their performance in unseen environments remains a challenge.Furthermore, compared to quality scores, fewer studies elaborate deep learning models to estimate intelligibility scores.This study proposes a multi-task speech intelligibility prediction model, called MTI-Net, for simultaneously predicting human and machine intelligibility measures.Specifically, given a speech utterance, MTI-Net is designed to predict human subjective listening test results and word error rate (WER) scores.We also investigate several methods that can improve the prediction performance of MTI-Net.First, we compare different features (including low-level features and embeddings from self-supervised learning (SSL) models) and prediction targets of MTI-Net.Second, we explore the effect of transfer learning and multi-tasking learning on training MTI-Net.Finally, we examine the potential advantages of fine-tuning SSL embeddings.Experimental results demonstrate the effectiveness of using cross-domain features, multi-task learning, and fine-tuning SSL embeddings.Furthermore, it is confirmed that the intelligibility and WER scores predicted by MTI-Net are highly correlated with the ground-truth scores.