Language-based audio retrieval with Converging Tied Layers and Contrastive Loss
Andrew Koh, Chng Eng Siong · 2022 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC) · 2022
In this paper, we tackle the new language-based audio retrieval task proposed in DCASE 202211https://dcase.community/challenge2022/task-language-based-audio-retrieval. Firstly, we introduce a simple, scalable architecture which ties both the audio and text encoder together. Our approach requires very minimal training, and allows us to use many publicly available models without needing to fine-tune them. Secondly, we show that using this architecture along with contrastive loss allows the model to beat the performance of the baseline model. Finally, in addition to having an extremely low training memory requirement, we are able to utilize pretrained models as it is without needing to finetune them. We test our methods and show that using a combination of our methods beats the baseline scores by 0.08 in R@1 and 0.13 in mAP10.