A Temporal Activity Graph Kernel for Human Activity Classification
Shobhanjana Kalita, Arindam Karmakar, Shyamanta Moni Hazarika · 2018
Human activities are predominantly spatio-temporal, involving spatial changes over time. Qualitative spatial relations between interacting entities are often used to describe spatial change. To derive such qualitative spatial relations, the interacting entities are approximated as a single bounding box or set of bounding boxes. A set of bounding boxes abstracting a single entity has been termed as an extended object; where each box is bounding a component. Extended object abstraction of spatial entities has been shown to be more effective for representation of human activities [10]. The temporal aspect of an activity is characterized through changing spatial relations between components of interacting extended objects over time. In this paper, we propose Temporal Activity Graph (TAG) based representation model to keep track of the sequences of relations between components of the extended objects. A kernel is designed for classification of spatio-temporal interactions in the TAG based model. The TAG kernel uses concepts of label sequence similarity and interestingness to compute similarity of a pair of TAGs. The TAG kernel is a generic solution that can be used with any kernel based method. Here, the kernel is used within a Support Vector Machine classifier. The TAG kernel based classification of activities is found on par with the state-of-the-art approaches for experiments performed on the Mind's Eye, the UT Interaction, and the SBU Kinect Interaction datasets.