Toward mmWave-Based Sound Enhancement and Separation
Muhammed Zahid Ozturk, Chenshu Wu, Beibei Wang, K. J. Ray Liu · ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · 2022
Speech enhancement and separation have been a long-standing problem with recent advances using a single microphone. With the help of video modality, improvements have been shown for these tasks. In this work, we explore a multimodal approach using mmWave radio devices, as these devices can measure vocal folds vibration. Thorough data collection and extensive experiments with two different neural networks indicate that radio modality can bring significant improvements in speech enhancement and separation.