A Deep-learning-based 3D Hand Pose Tracking System
Fan Yang · NAIST Digital Library (Nara Institute of Science and Technology) · 2018
Although the deep-learning-based hand pose estimation has been popular for quite a while, most of the existing works solely focus on the pose estimation model, while simply suppose the input, which is the depth image of the hand part, is given or can be directly acquired by a depth threshold.In a realistic situation, however, the complex foreground and background of the hand area may exist, and aforementioned methods may not be applicable.Hence, the goal of this work is to develop a deep-learning-based 3D hand pose tracking system, which can efficiently and robustly detect the hand from the raw depth image before estimating the 3D hand pose.It mainly includes three parts, as the hand detector, the hand verifier and the pose estimator.The hand detector generates a mask to segment the hand area from the raw depth image.If the hand verifier confirm the segmented hand is correct, the pose estimator generates corresponding 3D hand pose using the depth image covered by the mask.We evaluated our system on the tracking task of Hands In the Million (HIM2017) challenge and placed second.In addition, we also applied our modified tracking system on the object-interactive task of Hands In the Million (HIM2017) challenge and placed first.We find that using hand detector to segment hand from its interactive objects before performing pose estimation can make better results than directly performing pose estimation.