Segment-Level and Category-Oriented Network for Knowledge-Based Referring Expression Comprehension
Yuqi Bu, Xin Wu, Liuwu Li, Yi Cai, Qiong Liu, Qingbao Huang · 2023
Knowledge-based referring expression comprehension (KB-REC) aims to identify visual objects referred to by expressions that incorporate knowledge.Existing methods employ sentencelevel retrieval and fusion methods, which may lead to issues of similarity bias and interference from irrelevant information in unstructured knowledge sentences.To address these limitations, we propose a segment-level and category-oriented network (SLCO).Our approach includes a segment-level and promptbased knowledge retrieval method to mitigate the similarity bias problem and a categorybased grounding method to alleviate interference from irrelevant information in knowledge sentences.Experimental results show that our SLCO can eliminate interference and improve the overall performance of the KB-REC task.‡