Semi-supervised Text Classification with Temporal Ensembling
Rong Xiang, Shiqun Yin · 2021
Supervised learning algorithms require a lot of labeled data to train the model while obtaining labeled data for large datasets is costly and time consuming. Semi-supervised learning algorithms can jointly learning from labeled and unlabeled data. Thus, it is expected to solve the difficulty of labeling large data. This paper aims to propose a method which can train in a supervised fashion with labeled and unlabeled data simultaneously. To achieve the goal, we extend temporal ensembling which give a pseudo label for each example in unlabeled data into text classification. We propose a semi-supervised model which combine Bi-GRU and temporal ensembling. We use Bi-GRU as the deep neural network and use labeled data and unlabeled data to train Bi-GRU simultaneously. During training, pseudo-label is an ensemble prediction which aggregating the predictions of multiple previous network evaluations given by trained Bi-GRU. The experimental results demonstrate that our approach provides improvements when compared to the state of the art methods.