Hand Pose Detection in HMD Environments by Sensor Fusion using Multi-layer Perceptron
Luc Cong Vu, Bum-Jae You · 2019
The paper proposes a sensor fusion method to detect the pose of user's hand in head-mount display environments by using a Leap Motion Camera (LMC) with simple circular artificial markers on surfaces of a box wearing on the back of hand and two IMU sensors. One IMU sensor is located on the box and the other IMU sensor is fixed with the camera. Multilayer Perceptron (MLP) is adopted to transform the hand's pose in IMU coordinate system into the pose in global coordinate system attached at LMC by minimizing Mean Square Error (MSE) for Virtual Reality (VR)/Mixed Reality (MR) applications. The pose detection results are compared with poses of bare hand captured by LMC while the estimated data after transformation is fitted well with reference data in the sense that the average of mean difference for each roll, pitch and yaw angle is around 3.54 degree. It is applied successfully to track and estimate the pose of user's hand in around 70Hz.