Lifelong Visible-Infrared Person Re-Identification via a Tri-Token Transformer with a Query-Key Mechanism
Yitong Xing, Guoqiang Xiao, Michael S. Lew, Song Wu · 2024
Visible-infrared person re-identification has been extensively explored, but it typically relies on stationary datasets for training. However, data is collected in a streaming manner in practical scenarios, necessitating the model's ability to continually learn without forgetting previous tasks. Existing methods focus simply on the lifelong single-modality person re-identification, but the visible images are sometimes unavailable, e.g., at night. To this end, this paper introduces a more challenging yet practical problem: Lifelong Visible-Infrared Person Re-identification (LVI-ReID). Inspired by the complementary learning systems, we propose a Tri-Token transformer with a Query-Key mechanism (TTQK) to tackle the LVI-ReID. Firstly, a general token is designed to capture robust domain-general features, shared across different domains, aiming to enhance the generalization capability. Subsequently, recognizing that different domains possess unique features like illumination and scenes, we allocate a specific token for each domain to extract significant domain-specific features, aiming to enhance the adaptability across domains. Furthermore, to prevent using the task identifier in the inference stage of LVI-ReID, we design a query-key mechanism to adaptively select the appropriate specific token based on the similarity between the query token and keys. Extensive experiments demonstrate that our method outperforms other lifelong learning and LReID methods. The source code of our designed LVI-ReID method is at https://github.com/SWU-CS-MediaLab/TTQK.