Facilitating User Interaction with Complex Systems via Hand Gesture Recognition
Joshua New, Erion Hasanbelliu, Mario Aguilar · 2003
Ongoing efforts at our laboratory have been aimed at developing techniques that reduce the complexity of interaction between humans and computers. In particular, we have investigated the use of a gesture recognition system to support natural user interaction while providing rich information content. In this paper, we present a real-time gesture recognition system which can track hand movement, define orientation, and determine the number of fingers being held up in order to allow control of an underlying application. The system was developed in a modular fashion to maximize reutilization and portability. We utilized offthe-shelf image processing libraries and a low-cost web camera. Its main functional components consist of single-frame image processing for noise reduction, hand segmentation, arm removal, hand displacement and pose calculations, and a heuristic approach to finger-counting. In our particular implementation, the hand gesture information is used to control a medical image visualization application developed in our lab to illustrate multimodal information fusion. Here, a 3D model of a patient’s skull, generated from the patient’s MRI imagery, is directly manipulated by hand gestures. Within the application, the user can rotate the skull or zoom by reconfiguring or moving their hand in prescribed directions in front of the video camera. For example, moving the hand to the left or right of the camera’s field-of-view corresponds to yaw left or yaw right, respectively. By supporting a more natural interface modality and utilizing only common hardware and software components, human-computer interaction has been simplified while also enriched. Sample images, results, videos, and benchmark tests are presented.