A Robust Daily Human Activity Recognition and Prediction System
Md. Zia Uddin · 2008
In this work, a system is proposed for Human Activity Prediction (HAP) using activity sequence spanning tree constructed from a life-log created via a depth video camera-based daily Human Activity Recognition (HAR) approach using time-sequential augmented Local Binary pattern (LBP) and Enhanced Independent Component (EIC)-based depth silhouette features with Hidden Markov Models (HMMs). Regarding the daily HAR, the augmented local features are extracted first from the collection of the depth silhouettes containing various daily human activities such as walking, sitting, lying, cooking, neutral etc. Using these features, HMMs are used to model the time sequential information and recognize several activities. The depth silhouette-based activity recognition results show superior performance over the traditionally used binary silhouette-based approaches, The depth silhouette-based human activity recognition system can be used to recognize the activities automatically, which can be utilized to create life-log. In this regard, an easy prediction method is also proposed for predicting of activity in next few frames while testing a small video which may consist of multiple activities. It can save recognition time and help us to assign a long-time activity annotation for fast life-log construction. After building a life-log consisting of activity sequences, furthermore, a method for human activity prediction in long video is proposed using activity sequence spanning tree built from the activity sequence database. Based on the consecutive activities recorded in an activity sequence database (i.e. life-log) for a specific period of time of each day over a long time such as a month, the spanning tree can be constructed for the sequences starting with each activity where the leaf nodes contain the frequency of the consecutive activity sequences. Once the tree is constructed, to predict an activity after a sequence of activities, traverse the spanning tree until a path up to the previous node of the leaf nodes is matched with the testing pattern and finally, prediction of the next activity is done based on the highest frequency of the leaf nodes along the matched path. The prediction experiments over computer simulated data shows satisfactory results. The proposed video sensor-based human activity recognition and prediction systems can be utilized for important practical applications such as smart healthcare, proactive computing etc.