TextGCL: Graph Contrastive Learning for Transductive Text Classification
Yawei Zhao, Xiaoyang Song · 2023
Text classification is a fundamental task of natural language processing and has rich application scenarios. GNN models have recently attracted much attention due to their ability to encode complex graph-structured data. However, most of the works using GNNs for text classification only utilize static word vector models like word2vec and the encoding quality of node embeddings is mediocre, only few works explore the method of combining large-scale pretrained models with GNNs. Graph Contrastive Learning is a self-supervised learning method that can obtain high-quality node representations through pretraining without labeled data, which can be applied to downstream tasks such as node classification. In this work, we propose TextGCL, which introduces graph contrastive learning technique to transductive text classification task. TextGCL not only combines the ability of graph contrastive learning techniques to extract important information from graph structures, but also obtains the ability of large-scale pretrained models to extract rich information from text. In addition, transductive learning enables the model to jointly use unlabeled data (text only, no labels) and labeled data during the training stage, which greatly improves the performance of the entire model. We obtain state-of-the-art results on five widely used text classification benchmark datasets, empirically demonstrating the superiority of our model.