Performance Improvement Method of the Video Visual Relation Detection with Multi-modal Feature Fusion
Kwang‐Ju Kim, Pyong-Kun Kim, Kil-Taek Lim, Jong Taek Lee · 2022
Video visual relation detection is a novel research problem that aims to detect instances of visual relations of interest in a video. In this paper, we propose a performance improvement method of the video visual relation detection with multi-modal feature fusion. First, we introduce a spatial feature extraction method that is designed to include the relative positions of objects itself and between objects in the image. Next, we suggest a relationship classifier that is designed to accommodate the complexity of the input features. Our proposed method achieves 6.65 mAP, and ranked the 2nd place in the visual relation detection task of Video Relation Understanding Challenge (VRU), the ACM Multimedia 2020.