Lightweight Transformer for sEMG Gesture Recognition with Feature Distilled Variational Information Bottleneck
Zefeng Wang, Bingbing Hu, Junfeng Yao, Jinsong Su · Frontiers in artificial intelligence and applications · 2024
Gesture recognition based on surface electromyography (sEMG) has seen considerable improvements in performance across various tasks and metrics with the rapid development of deep learning. However, challenges still exist in current deep neural networks for sEMG recognition. For instance, convolutional neural networks exhibit poor capturing of global features, recurrent neural networks have limited parallel processing capabilities, and their hybrids are usually more complex. Additionally, recent networks based on Transformers rarely consider the locality of attention and noise resistance. To fully explore the essence of sEMG sequences, and to make the model more lightweight and robust while ensuring feature learning performance, in this paper, we propose the feature distilled variational information bottleneck (FDVIB). Specifically, this method leverages knowledge distillation to learn from a high-precision teacher model at levels of feature and prediction, significantly reducing parameters and computations, and simplifying the structure. It also uses VIB to enhance the model’s robustness. We construct a Transformer model using the proposed method and conduct a series of evaluations. Experimental results show that our classification accuracy is competitive with state-of-the-art and also demonstrate the effectiveness of our method in enhancing model lightness and robustness.