Resnet-Conformer Network using Multi-Scale Channel Attention for Sound Event Localization and Detection in Real Scenes
Lihua Xue, Hongqing Liu, Yi Zhou, Lu Gan · 2023
Sound Event Localization and Detection (SELD) aims at detecting the class and activity time of the sound events, while estimating their corresponding direction of arrival (DOA). The difficulties of SELD lie in the change of acoustic scenes and the presence of overlapping sound events. To improve the performance of SELD, in this work, we propose a Resnet-Conformer network based on multi-scale channel attention (MSCA-RCnet) to conduct SELD. To that end, first, we take the RCnet as the backbone of the model and add the multi-scale channel attention (MSCA) to the Resnet block to capture channel-level dependencies. Second, we utilize attentive statistics pooling (ASP) to resolve the mismatch between label resolution and frame-level feature temporal resolution and enhance the discriminability of the features. Finally, we integrate the MSCA-RCnet and the Event-Independent Network V2 (EINV2) by multi-ACCDOA representation into an EINV2-based multi-ACCDOA network. Subsequently, we perform an average ensemble of these two models. To address the shortage of training data, we employ audio channel swapping (ACS) and AugMix to the dataset. Finally, we evaluate our proposed systems in the Audio-only Track of the DCASE 2023 Task 3 and rank the second place in the competition.