Interaction Recognition in a Paired Egocentric Video
Ahad Khatri · RIT Scholar Works (Rochester Institute of Technology) · 2020
Wearable devices and affective computing have gained popularity in the recent times. Egocentric videos recorded using these devices can be used to understand the emotions of the camera wearer and the person interacting with the camera wearer. Emotions affect the facial expression, head movement and various other physiological factors. In order to perform this study we collected dyadic conversations (dialogues between two people) data from two different groups; one where two individuals agree on certain topic and second where two individuals disagree on certain topics. This data was collected using a wearable smart glass for video collection and a smart wristband for physiological data collection. Building this unique dataset was one of the significant contributions of this study. Using this data we extracted various features that include Galvanic Skin Response (GSR) data, facial expressions and 3D motion of a camera within an environment which is termed as Egomotion. We built two different machine learning models to model this data. In the first approach we use an application of Bayesian Hidden Markov model for classifying these individual videos from the paired conversations. In the second approach we use a Random Forest classifier to classify the data based on the Dynamic Time Warping data between the paired videos and individual average data for all the features in individual videos.The study found that in the presence of the limited data used in this work, individual behaviors were slightly more indicative of the type of discussion (85.43% accuracy) than the coupled behaviors (83.33% accuracy).