Sparsity-Driven Gesture Recognition Using Lightweight TCNFormer Networks in Millimeter-Wave Radar
Hao Wu, Biao Jin, Zhenkai Zhang, Zhuxian Lian, Baoxiong Xu, Jin Liang, Xiangqun Zhang, Genyuan Du · IEEE Transactions on Instrumentation and Measurement · 2025
Gesture recognition with millimeter-wave radar has broad application prospects in human-computer interaction. However, traditional recognition methods generate overly redundant features and construct large-scale networks, rendering them unsuitable for embedded devices with limited memory. To address this challenge, we propose a sparse-driven dynamic gesture recognition network in millimeter-wave radar, named TCNFormer (Time Convolutional Network and transFormer). Firstly, we employ a two-dimensional fast Fourier transform (2D-FFT) to obtain the range-Doppler maps (RDM). These maps are then processed through incoherent integration of multiple frames to produce Doppler-time maps (DTM). We subsequently use the orthogonal matching pursuit algorithm to achieve a sparse representation of the Doppler-time trajectories and integrate the RDM to extract the range features of gestures, obtaining the multi-dimensional sparse sequences encompassing the range-Doppler-time feature. We then design a TCNFormer network tailored to these multi-dimensional sparse sequences. This network leverages a shallow TCN to learn local features, a Transformer network to capture global features and an adaptive weighting method to fuse these local and global features effectively. Experimental results demonstrate that our network fully exploits the sparse multi-dimensional sequences, achieving a recognition accuracy of 99.17% on a self-built dataset. The parameter size of the network is only 0.13M, significantly outperforming existing state-of-the-art models in relevant metrics, thereby proving its suitability for embedded applications in human-computer interaction.