A Comprehensive Study of Speech Separation: Spectrogram vs Waveform Separation
Fahimeh Bahmaninezhad, Jian Wu, Rongzhi Gu, Shi-Xiong Zhang, Yong Xu, Yu Meng, Dong Yu · 2019
Speech separation has been studied widely for single-channel close-talk microphone recordings over the past few years; developed solutions are mostly in frequency-domain.Recently, a raw audio waveform separation network (TasNet) is introduced for single-channel data, with achieving high Si-SNR (scale-invariant source-to-noise ratio) and SDR (sourceto-distortion ratio) comparing against the state-of-the-art solution in frequency-domain.In this study, we incorporate effective components of the TasNet into a frequency-domain separation method.We compare both for alternative scenarios.We introduce a solution for directly optimizing the separation criterion in frequency-domain networks.In addition to speech separation objective and subjective measurements, we evaluate the separation performance on a speech recognition task as well.We study the speech separation problem for far-field data (more similar to naturalistic audio streams) and develop multi-channel solutions for both frequency and time-domain separators with utilizing spectral, spatial and speaker location information.For our experiments, we simulated multi-channel spatialized reverberate WSJ0-2mix dataset.Our experimental results show that spectrogram separation can achieve competitive performance with better network design.Multi-channel framework as well is shown to improve the single-channel performance relatively up to +35.5% and +46% in terms of WER and SDR, respectively.