CASQ: Enhancing Human-Object Interaction Detection via Supplementary Semantic Information for Interaction Queries
Thinh V. Le, Huyen Ngoc N. Van, Nguyen D. Vo, Khang Tan Tran Minh Nguyen · 2023
Human-Object Interaction (HOI) detection, which involves identifying and recognizing the interactions between humans and objects in an image, has garnered great interest from the computer vision research community. However, existing HOI detection methods are limited in terms of accuracy due to the use of static interaction queries in the inference phase. To address this issue, we propose a novel method that utilizes supplementary semantic information to generate dynamic interaction queries per image. Our method involves embedding object categories into vector space using a pre-trained CLIP model and incorporating attention information from the semantic features, which enhances its representation and query capabilities. Our proposed CASQ significantly improves the accuracy and performance of HOI detection, accounting for variations in context and characteristics of the interaction. We evaluated CASQ on the standard V-COCO benchmark dataset and demonstrate that it outperforms existing methods, achieving state-of-the-art results.