A Robust Multi-Frame mmWave Radar Point Cloud-Based Human Skeleton Estimation Approach with Point Cloud Reliability Assessment
Xintong Shi, Tomoaki Otsuki Ohtsuki · 2023
Millimeter-Wave (mmWave) radar-based skeleton estimation has gained significant attention in the field of human motion analysis and sensing. It offers distinct advantages over RGB camera-based, depth camera-based, and inertial sensor-based approaches. It operates independently of lighting conditions. Additionally, it overcomes limitations of depth camera-based methods such as reflections and occlusions, and provides precise real-time tracking, enhancing its overall performance. However, existing human skeleton estimation methods utilizing point cloud data mostly rely on single-frame inputs or incorporate voxelization, which introduces drawbacks. This paper proposes a novel Convolutional Neural Network (CNN) and Bi-directional Long Short-Term Memory (BiLSTM)-based model for multi-frame point cloud data without voxelization. Furthermore, an Long Short-Term Memory (LSTM)-based neural network is introduced to assess point cloud reliability, enhancing robustness. Experimental results demonstrate improved accuracy and robustness in human skeleton estimation compared with the conventional single frame-based methods.