Bring Adaptive Binding Prototypes to Generalized Referring Expression Segmentation
Weize Li, Zhicheng Zhao, Haochen Bai, Fei Su · IEEE Transactions on Multimedia · 2025
Referring Expression Segmentation (RES), which aims to identify and segment objects based on natural language expressions is garnering increased research attention. While substantial progress has been made in RES, the emergence of Generalized Referring Expression Segmentation (GRES) introduces new challenges by allowing the expressions to describe multiple objects or lack specific object references. Existing RES methods usually rely on sophisticated encoder-decoder and feature fusion modules, and have difficulty generating class prototypes that match each instance individually when confronted with the complex referent and binary labels of GRES. In this paper, reevaluating the differences between RES and GRES, we propose a novel Model with Adaptive Binding Prototypes (MABP) that adaptively binds queries to object features in the corresponding region. It enables different query vectors to match instances of different categories, or different parts of the same instance, significantly expanding the decoder's flexibility, dispersing global pressure across all the queries, and easing the demands on the encoder. The experimental results demonstrate that MABP significantly outperforms the state-of-the-art methods in all three splits on the gRefCOCO dataset. Moreover, MABP outperforms the state-of-the-art methods on the RefCOCO+ and G-Ref datasets, and achieves very competitive results on RefCOCO. The code is available athttps://github.com/buptLwz/MABP.