Recognition of Actions and Subjects from Inertial and FSR Sensors Attached to Objects
Yikai Peng, Peter Jančovič, Martin J. Russell · 2020
This paper describes automatic systems for human activity recognition and identification of subjects based on sensorised objects. The action recognition system is based on a deep neural network - hidden Markov model (DNN-HMM) with augmentation of feature vectors by the i-vector of a given recording to deal with the subject variability. The subject identification system is based on i-vectors. The sensors, comprising an accelerometer, gyroscope, magnetometer and force-sensitive resistors (FSRs), are packaged in a coaster attached to the base of an object, here a jug. Evaluations are performed using nearly 11 hours of data recordings from 26 subjects, containing actions involved in manipulating a jug to make cups of tea. We demonstrate the performance of the DNN-HMM action recognition system in a subject-dependent and subject-independent case and in controlled and natural scenarios. While the subject-dependent system achieved error rate of 0.15% in controlled scenario, this increased up to 15.33% for subject-independent system in natural scenario. The proposed i-vector augmentation provided 26% relative error rate reduction. The subject identification system based on i-vectors and cosine similarity calculation achieved over 68% recognition accuracy in natural scenario.