Camera-based Hand Tracking using a Mirror-based Multi-view Setup
Guan Ming Lim, Prayook Jatesiktat, Christopher Wee Keong Kuah, Wei Tech Ang · 2020
Current clinical practice of measuring hand joint range of motion relies on a goniometer as it is inexpensive, portable, and easy to use, but it can only measure the static angle of a single joint at a time. To measure dynamic hand motion, a camera-based system that can perform markerless hand pose estimation is attractive, as the system is ubiquitous, low-cost, and non-contact. However, camera-based systems require line-of-sight, and tracking accuracy degrades when the joint is occluded from the camera view. Thus, we propose a multi-view setup using a readily available color camera from a single mobile phone, and plane mirrors to create multiple views of the hand. This setup eliminates the complexity of synchronizing multiple cameras and reduce the issue of occlusion. Experimental results show that the multi-view setup could help to reduce the error in measuring the flexion angle of finger joints. Dynamic hand pose estimation with object interaction is also demonstrated.