Self-supervised Pre-trained Contributes to Few-shot Wake Word Detection
Chen Chen, Xin Lin, Xiangxian Zhu, Xudong Lou · 2024
Wake words or keywords, often brief vocalizations or control phrases, typically last around 1 second. These audio signals undergo Short-Time Fourier Transform (STFT) and Mel Frequency Cepstral Coefficients (MFCC) transformations, resulting in two-dimensional signal graphs. Employing Convolutional Neural Networks (CNNs) treating them as single-channel images facilitates efficient feature extraction, enhancing the keyword spotting task. While models proficiently recognize a predefined set of samples based on the keyword categories in the dataset (closed-set classification), the challenge lies in extending this capability to custom keyword spotting. In the latter scenario, the model must identify keywords from previously unseen categories, necessitating a system-wide capability for personalized keyword customization. One approach involves recording new keywords, retraining the model, and synchronizing it through methods like Over-the-Air (OTA) updates. However, in cost-efficient environments, offline model training poses challenges, especially when dealing with newly added keywords, often accompanied by only a limited number of samples. Existing methods struggle to achieve effective custom keyword spotting with such constrained data. This paper tackles these challenges by emphasizing representation learning and few-shot classification. It commences with self-supervised learning to pretrain the feature encoder, separately validating the Momentum Contrast version 3 (MoCov3) and Masked Autoencoder (MAE) pre-training methods to ensure high-quality feature representations. In the classification phase, few-shot learning is employed for open-set prediction, incorporating Triplet Loss for calculating sample similarity and deploying the Prototypical Networks to compute mean vector representations for prototype categories. Experimental results robustly demonstrate that the proposed method for few-shot custom keywords spotting surpasses existing approaches.