Speaker-Aware Speech Enhancement with Self-Attention
Ju Lin, Adriaan J. van Wijngaarden, Melissa C. Smith, Kuang‐Ching Wang · 2021 29th European Signal Processing Conference (EUSIPCO) · 2021
Speech enhancement aims to improve the intelligibility and quality of speech that is affected by noise. In this paper, we propose a novel speaker-aware speech enhancement (SASE) method that extracts speaker information using long short-term memory (LSTM) layers, and then uses a convolutional recurrent neural network (CRN) to embed the extracted speaker information. It is shown in a series of comprehensive experiments that only a few seconds of reference audio suffice for the proposed SASE method to perform better than LSTM and CRN baseline systems. The addition of a self-attention mechanism can further boost relevant speech-quality metrics.