Human-Object Interaction Detection: 1D Convolutional Neural Network Approach Using Skeleton Data
Plamen Hristov, D.R. Avresky, Ognian L. Boumbarov · 2021
Human-object interaction detection is a somewhat recently emerged scientific topic, which is mainly due to the advent of deep learning algorithms. Most current methods are performed on single images, detecting separately humans and objects, using state-of-the-art pose detection and object detection networks. The networks ease the overall task by allowing for learning of the readily inferred features. When adding the time dimension into the equation, this task becomes more complex, as temporal features between frames have to be taken into account. The paper aims to show an approach for detecting human interactions in videos, which utilizes several different methods - YOLOv5 for object detection, CSR-DCF and Kalman Filter for object tracking, and ID Convolutional Neural Network (1D-CNN) for real-time interaction detection. The overall algorithm is purposed for salient and rigid (modern-solid) objects in mind, positioned in closed-door scenes. The dataset and task are privately defined, that is they are relevant to this work only and cannot be compared to other works. The overall algorithm is tested on a subset of the PKU Multi-Modality Dataset (PKUMMD).