Dynamically Contrastive Clustering For Sentence Embedding
Haidong Shi, Xiaotang Zhou, Lingbo Li · 2024
U nsupervised contrastive learning to obtain sentence embedding has become a widely adopted and effective method. However, existing techniques mainly focus on the construction of positive-negative sample pairs, where instance contrastive learning is performed on samples by determining the positive and negative examples, while these methods ignore the effect of temperature coefficients on the model and tend to use fixed temperature coefficients. To address these issues, we propose a new approach named DCCSE(Dynamically Contrastive Cluster for Sentence Embedding), our model smooths the sample embedding while enhancing the sample linkage. Specifically, we use sample embeddings to dynamically update the cluster embeddings, combine contrastive learning and cluster centroid while employing cluster embed dings to smooth the sample embeddings and design a dynamic temperature coefficient function to improve the linkage between sample pairs. We evaluate our approach on a standard STS dataset, and the experimental results show that our model has relatively significant advantages over other baseline models.