Dynamic Scene Semantic Information Guided Framework for Human Motion Prediction in Proactive Human-Robot Collaboration
Pengfei Ding, Jie Zhang, Peng Zhang, Geng Li · 2023
Human motion prediction plays an important role in proactive human-robot collaboration. However, current research ignores scene semantic information, especially the dynamic information, which is closely related to future human motion, and therefore the prediction is not accurate enough. To address the above issue, we propose a dynamic scene semantic information guided framework for human motion prediction in proactive human-robot collaboration. We first ascertain the collaboration space depending on the gaze point and recognize the object category. After obtaining the static information, we detect object affordance, which represents the property of objects, and estimate object 6-DoF pose to gain the dynamic information with object manipulation analysis. Then, we design a diffusion sampling strategy to obtain diverse future human motion results. Simultaneously, to ensure the reliability of the prediction, a step-by-step driven ST-GCN with regulatory mechanism is devised. The results of experiment demonstrate the effectiveness of our method in human-robot collaboration on an assembly task.