Dynamic Margin-based Contrastive Learning for Robust Negative Sampling in Information Retrieval
Tsai-Tsung Chen, Chuan‐Ju Wang, Ming-Feng Tsai · 2025
Modern information retrieval (IR) systems, powered by bi-encoder architectures and pretrained language models, rely on effective negative sampling for contrastive learning. While easy negatives are computationally simple, they fail to challenge the model, whereas hard negatives-selected via methods like BM25, ANCE, or ADORE-can be overly difficult and misleading. To this end, this paper proposes dynamic margin-based contrastive learning (DMCL), which adaptively adjusts the decision boundary based on query-negative similarity, ensuring consistent exposure to moderately hard negatives. Experiments across diverse datasets and models show that DMCL outperforms traditional methods, achieving state-of-the-art retrieval performance with minimal computational cost.