QRFPose: Query-Based 3D Pose Estimation using Radio Signals
Hong Wan, Ruiyuan Song, Chunyang Xie, Zhi Peng Lu, Qi Chen, Zhi Wu, Dongheng Zhang, Yang Hu, Yan Chen · 2024
The major challenge for radio-frequency-based (RF) multi-person pose estimation is to solve the problem of the global information-sparsity and local information-dense in RF signals. Unlike the visual image that the human body is located in a determined region, the majority of information regarding the human body in RF signals is concentrated in localized areas, with a small portion dispersed across global regions. However, the existing works directly follow the divide-and-conquer strategy to detect candidate regions where people may exist, and then zoom into each local region to generate human skeletons, while the information beyond local information-dense regions is ignored resulting in performance degradation. To resolve the problem, we propose a novel query-based multi-person pose estimation framework, QRFPose, to view body joints as learnable query embeddings to adaptively focus on features related to target skeleton joints. Moreover, QRFPose is carefully designed to take the geometric prior of horizontal and vertical RF signals into consideration. By leveraging the projective deformable attention rather than the naive global attention mechanism, QRFPose works more efficiently. The experimental results show that QRFPose plays favorably against state-of-the-art approaches in terms of both accuracy and efficiency.