Global Shapes and Salient Joints Features Learning for Skeleton-Based Action Recognition
Xingtong He, Xu Liu, Licheng Jiao · IEEE Signal Processing Letters · 2023
Global shapes and local joints are significant cues to learn skeleton representations for human action recognition. However, most shapes-based methods ignore local joints information and thus perform less well when recognizing similar actions. In this letter, we propose a two-stream method of global shapes and salient joints features learning to address the above issues. Firstly, the global sparse shapes modeling stream (GSSM) explores the global skeletal shapes in Kendall space and codes the nonlinear shapes by a sparse coding and dictionary learning method. Then, the salient enhancing joints generation stream (SEJG) explores local joints information by extracting discriminative salient joints and generates frame-enhancing action sequences with invariant length by a bilinear frame interpolation module. To improve the temporal modeling ability of our method, we use similar multi-scale LSTM methods in both GSSM and SEJG, which explore global-local features in static, short-term, and long-term temporal scales. Extensive experiments on four challenging datasets verify the effectiveness of our proposed method, which achieves competitive results compared to state-of-the-art (SOTA) methods.