An Improved Semi-supervised Variational Autoencoder with Gate Mechanism for Text Classification
Haiming Ye, Weiwen Zhang, Mengna Nie · International Journal of Pattern Recognition and Artificial Intelligence · 2022
In recent years, semi-supervised learning has been investigated to take full advantages of increasing unlabeled data. Although pretrained deep learning models are successfully adopted on a massive amount of unlabeled data, they may not be applicable in specific domains as the data is limited. In this paper, we propose a model, termed Semi-supervised Variational AutoEncoder (SVAE), which consists of Gated Convolutional Neural Networks (GCNN) as both the encoder and the decoder. Since the canonical VAE suffers from Kullback–Leibler (KL) vanishing problem, we attach a layer named Scalar after Batch Normalization (BN) to scale the output of the BN. We conduct experiments on two domain-specific datasets with a small amount of data. The results show that SVAE outperforms other alternative baselines for language modeling and semi-supervised learning studies. Especially, the results in the language modeling validate the effect of combining BN and Scalar for tackling the KL vanishing problem. Moreover, the visualization of the latent representations verifies the performance of SVAE on less data.