A Hand Action Recognition Framework Based on Multimodal Foundation Model and Tracking of Skeleton Point Trajectorie
Wu Liang, Xiaoyu Jiang, Li Ma · 2024
Although accurate hand motion recognition was particularly crucial in monitoring modern industrial assembly lines, it still faced challenges, such as high-cost training datasets and poor real-time performance. In this paper, we propose an innovative paradigm for recognizing hand actions on industrial assembly lines, wherein the problem of hand action recognition was transformed into a hand skeletal point trajectory classification problem. This solved the problem of the requirement for a large dataset and poor real-time performance for action recognition on the assembly line. Comprehensive experiments were concluded to demonstrate the effectiveness and superiority of the method, wherein the accuracy of hand action recognition reached 98.8% with the proposed methods being implemented in real industrial assembly lines of Midea Group.