Natural user Interfaces for Mixed Reality: Controlling Virtual Objects with your Real Hands
Sérgio Serra, Redouane Kachach, Ester González-Sosa, Álvaro Villegas · 2020 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW) · 2020
We explore a novel way of interacting in Mixed Reality using jointly hand dynamics and appearance. Our prototype is based on HTC Vive, Leap Motion sensor and ZED Mini stereo camera. Leap Motion allows to track the egocentric hands in 3D. To get the segmented hands, a color-based algorithm is applied to the egocentric images. A user interface consisting on interactive buttons is designed, one with physical props enabling haptic sensation. Tracking of physical objects is implemented using fiducial marker and Aruco library. Results suggest that users find attractive this novel interactive way, specially while using physical prop.