Manipulation of virtual objects in marker-less AR system by fingertip tracking and hand gesture recognition
Byungsung Lee, Junchul Chun · 2009
In this paper, we present a vision-based approach to control the augmented reality (AR) objects robustly in a marker-less AR system. It is known that one of the promising ways to develop a marker-less AR system is using human's body such as hand or face for replacing fiducial markers. This paper presents a real-time method to manipulate the AR objects dynamically in a developed 3D hand coordinate system. To build the marker-less system, we utilize YCbCr skin-color model for hand shape detection and curvature-based fingertip detection from an input video images. Using the detected fingertips the camera pose are estimated to register virtual objects on the hand coordinate system. In order to control the movements of the virtual objects rendered on the marker-less AR system dynamically, a vision-based hand control interface, which exploits optical flow for the movement of the objects and pattern recognition for the hand command initiation, is developed. From the experiments we can prove that the proposed hand mouse can manipulate the objects in a feasible fashion.