Leveraging Self-Supervised Learning for Enhanced Image Classification
Kaiyuan Xu, Zhao Hui · 2024
This study investigates the effectiveness of self-supervised learning (SSL) strategies, including SimCLR and MoCo, in image classification tasks, with a focus on large-scale datasets such as ImageNet. We evaluate SSL methods using varying proportions of labeled data on ImageNet and compare their performance to traditional supervised learning models such as AlexNet and VGG16. Experimental results show that the combination of SimCLR with Vision Transformer (ViT) achieved a Top-1 accuracy of 78.5%, surpassing SimCLR with ResNet (76.3%) and MoCo with ViT (77.3%), demonstrating superior generalization on ImageNet. Hyperparameter tuning, including adjustments to batch size, learning rate, and temperature, significantly improved model performance, resulting in more stable loss curves. Feature visualization using t-SNE further demonstrated that SSL models like SimCLR and MoCo achieved better class separation and intra-class cohesion than traditional supervised learning models. Computational cost analysis revealed that while ViT provided superior performance, its high computational demands make it less feasible in resource-limited environments. In contrast, SimCLR showed more computational efficiency, making it preferable for large-scale datasets. These findings highlight the scalability and robustness of SSL methods, suggesting that future work should focus on optimizing SSL for class-imbalanced datasets and exploring more efficient architectures for real-world applications.