A survey of contrastive learning in NLP
Haolin Sun, Jie Liu, Jing Zhang · 7th International Symposium on Advances in Electrical, Electronics, and Computer Engineering · 2022
Contrastive learning (CL) is building example pairs and computing loss to make models more robust in processing samples by reducing the distance between positive samples and amplifying the distance between negative samples. Recently, CL has attracted interest in natural language processing (NLP) where it has worked well and is mainly applied to sentence embedding and text classification tasks. To our knowledge, no study has reviewed the application of CL in NLP. In this paper, we describe two types of contexts with CL and present the methods to compute different losses. And then, we introduce some classic models which are significant. Finally, we discuss the current challenges and possible future directions of the CL.