Inflate and Shrink:Enriching and Reducing Interactions for Fast Text-Image Retrieval
Haoliang Liu, Tan Yu, Ping Li · Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing · 2021
By exploiting the cross-modal attention, cross-BERT methods have achieved state-of-the-art accuracy in cross-modal retrieval.Nevertheless, the heavy text-image interactions in the cross-BERT model are prohibitively slow for large-scale retrieval.Late-interaction methods trade off retrieval accuracy and efficiency by exploiting cross-modal interaction only in the late stage, attaining a satisfactory retrieval speed.In this work, we propose an inflating and shrinking approach to further boost the efficiency and accuracy of late-interaction methods.The inflating operation plugs several codes in the input of the encoder to exploit the text-image interactions more thoroughly for higher retrieval accuracy.Then the shrinking operation gradually reduces the text-image interactions through knowledge distilling for higher efficiency.Through an inflating operation followed by a shrinking operation, both efficiency and accuracy of a late-interaction model are boosted.Systematic experiments on public benchmarks demonstrate the effectiveness of our inflating and shrinking approach.