A view-based multiple objects tracking and human action recognition for interactive virtual environments
Jin‐Woo Choi, Yong-Il Cho, Kyusung Cho, Sujung Bae, Hyun Seung Yang · 2008
Abstract—As environments become smart in accordance with advances in ubiquitous computing technology, researchers are struggling to satisfy users ’ diverse and sophisticated demands. The aim of the present work is to enable multiple persons in a interactive virtual environment to simultaneously and conveniently interact with virtual agents. To this end, we propose a real-time system that robustly tracks multiple persons in virtual environments and recognizes their actions through image sequences acquired from a single fixed camera. The proposed system is compromised of three components: blob extraction, object tracking, and human action recognition. Given an image, we extract blobs using the Mixture of Gaussians algorithm with a hierarchical data structure and we additionally remove shadows and highlights in order to obtain a more accurate object silhouette. We then track multiple objects using a motion-based object model and an inference graph for handling grouping and fragment problems. Finally, we model an action as a Motion History Image (MHI) based on given object tracks, normalize the MHI, reduce the MHI using PCA, and classify an action using a multi-layer perceptron. To evaluate the performance of the proposed system, we employed it in an augmented reality application where multiple persons can interact with a virtual pet. The results confirm that reliable object tracking is achieved and multiple persons ’ actions can be recognized for applications in interactive virtual environments. Index Terms—Human action recognition, human computer interaction, object detection, object tracking. I.