A universally-deployable ASR frontend for joint acoustic echo cancellation, speech enhancement, and voice separation
Thomas R. O'Malley, Arun Narayanan, Quan Wang · Interspeech 2022 · 2022
Recent work has shown that it is possible to train a single model to perform joint acoustic echo cancellation (AEC), speech enhancement, and voice separation, thereby serving as a unified frontend for robust automatic speech recognition (ASR).The joint model uses contextual information, such as a reference of the playback audio, noise context, and speaker embedding.In this work, we propose a number of novel improvements to such a model.First, we improve the architecture of the Cross-Attention Conformer that is used to ingest noise context into the model.Second, we generalize the model to be able to handle varying lengths of noise context.Third, we propose Signal Dropout, a novel strategy that models missing contextual information.In the absence of one or more signals, the proposed model performs nearly as well as task-specific models trained without these signals; and when such signals are present, our system compares well against systems that require all context signals.Over the baseline, the final model retains a relative word error rate reduction of 25.0% on background speech when speaker embedding is absent, and 61.2% on AEC when device playback is absent.