Drone Ego-Noise Cancellation for Improved Speech Capture using Deep Convolutional Autoencoder Assisted Multistage Beamforming
Yanjue Song, Stijn Kindt, Nilesh Madhu · 2022 25th International Conference on Information Fusion (FUSION) · 2022
We propose a multistage approach for enhancing speech captured by a drone-mounted microphone array. The key challenge is suppressing the drone ego-noise, which is the major source of interference in such captures. Since the location of the target is not known a priori, we first apply a UNet-based deep convolutional autoencoder (AE) individually to each microphone signal. The AE generates a time-frequency mask ∈ [0, 1] per signal, where high values correspond to time-frequency points with relatively good signal-to-noise ratios (SNRs). The masks are pooled across all microphones and the aggregated mask is used to steer an adaptive, frequency domain beamformer, yielding a signal with an improved SNR. This beamformer output, after being fed back to the AE, now yields an improved mask - which is used for re-focussing the beamformer. This combination of AE and beamformer, which can be applied to the signals in multiple ‘passes' is termed multistage beamforming. The approach is developed and evaluated on a self-collected database. For the AE - when used to steer a beamformer - a training target that preserves more speech at the cost of less noise suppression outperforms an aggressive training target that suppresses more noise at the cost of more speech distortion. This, in combination with max-pooling of the multi-channel mask - which also lets through more speech (and noise) compared with median pooling - performs best. The experiments further demonstrate that the multistage approach brings extra benefit to the speech quality and intelligibility when the input SNR is 2:-10 dB, and yields comprehensible outputs when the input has a SNR above -5 dB.