Range Camera for Simple behind Display Interaction
Anton Treskunov, Seung Wook Kim, Stefan Marti · 2011
This paper presents a method to estimate hand positions behind a display, while the person’s body is in front of the display. This allows for direct and bare hand interaction with virtual objects in an AR setting. As a sensor, we use a time-of-flight range camera, which provides depth information per pixel, but is noisy. While hand tracking has previously been attempted by localizing body parts and fitting a body model into the data, we propose a simpler algorithm. We first remove outliers, and then model the hand as an oriented box whose position and orientation is determined by a PCA algorithm. The calculated hand model is then used in a physics engine based interaction with virtual content. rendering on a transparent display), and interacts with a spatial environment by reaching with his hand behind the display. That way, the hand is spatially colocated with real and virtual content and, at the same time, does not obscure the display surface. 1.