Multi-views contrastive learning for dense text retrieval
Yang Yu, Jun Zeng, Lin Zhong, Min Gao, Junhao Wen, Yingbo Wu · Knowledge-Based Systems · 2023
Dense text retrieval has become a widely used paradigm for recalling existing language models , and efficient dense text retrieval is essential for obtaining sufficiently accurate candidate samples. However, the existing methods for dense text retrieval, which typically use dual-encoder architectures to match similar queries and documents, suffer from a lack of information interaction at low data volumes, resulting in suboptimal performance. Additionally, existing research relies on negative sampling techniques that do not take into account the negative effects of single negative sampling bias on the robustness of the model. These limitations hinder the development of more robust dense text retrieval models . In this paper, we propose a multi-view contrast learning architecture, named MvCR, to address these issues. MvCR improves the performance of dense text retrieval by performing contrast learning with multiple views while significantly increasing the model’s ability to discriminate between positive and negative samples. Additionally, we propose a data augmentation method that focuses on increasing the number of hard negative samples with accurate and semantic matching features. The experimental results have shown that MvCR can perform as well as strong baseline models even when the data volume is small. Furthermore, MvCR achieved better results on two popular retrieval benchmarks with comparable amounts of data. Specifically, MRR@10 was 39 . 1 ( + 0 . 9 % ) and Recall@50 was 87 . 8 ( + 1 . 4 % ) on the MS-MARCO dataset. And Recall@5 increased to 77 . 2 ( + 1 . 8 % ) and Recall@50 increased to 85 . 3 ( + 1 . 0 % ) on the Natural Questions dataset.