Stroke Patterns Guided End-to-End In-Air Handwriting Recognition Using mmWave Radar
Yaoxi Chen, Qin Chen, Yü Tian, Yiming Pi, Zongjie Cao · IEEE Sensors Journal · 2025
In the task of in air handwriting recognition, millimeter-wave radar sensors offer advantages in low power consumption, privacy protection, and robustness to environmental conditions. The traditional approach is to convert the radar echo signals into radar images by digital signal processing (DSP) algorithms, and then perform trajectory tracking or recognition. In this paper, we propose an innovative end-to-end recognition model that directly processes raw radar signals without designing specific DSP algorithms. In order to solve the problem of network training difficulties caused by high dimensional raw radar signals, we introduce a multi-modal feature alignment method based on variational analysis, which utilizes the common stroke pattern of handwritten trajectories to guide network training. Specifically, the method employs 2D handwriting trajectory sequences to represent stroke patterns. Through the designed multi-modal feature alignment algorithm, the raw signal features extracted by the end-to-end network gradually converge to the easily accessible 2D handwriting trajectory features. Comparison experiments with traditional methods in complex handwriting recognition tasks demonstrate the superiority of the proposed method. Subsequent visualization analysis and ablation experiments further confirm the validity and interpretability of the model modules.