Self-Supervised Learning Using Noisy-Latent Augmentation
Rasa Khosrowshahli, Shahryar Rahnamayan, Azam Asilian Bidgoli, Masoud Makrehchi · 2023
Generally speaking, labeled data is difficult and expensive to provide for applications in machine learning and data mining. One of the earliest approaches to tackle this problem is semi-supervised self-training to take advantages of labeled and unlabeled data to create pseudo-labeled data. However, reaching a high level of confidence to predict the unseen data by a classifier with a limited number of labeled samples is a challenging task. Generating pseudo-labeled data can collaboratively improve the self-training to increase its confidence for further predictions. This paper proposes a collaborative framework between augmentation and self-training to accurately train a model with very limited labeled data. Our framework includes two components where the first component is an unsupervised technique to augment labeled data and feed to the second component which is self-training. The first component uses a custom variational autoencoder architecture (VAE) to generate new samples by adding randomly generated noise to encoded latent representation. As a result, the augmentation component can generate the unique and unexplored images with respect to limited input data distribution. We evaluated the proposed framework on handwritten MNIST image dataset. The conducted experiment shows that the generative component can be helpful in overcoming the problem of inaccurate self-training prediction when sufficient labeled data is not accessible.