Exploring Few-Shot Performance of Self-Supervised Visual Representations
Ved Prakash Upadhyay, Shivani Modi, Saloni Ajay Gupta, Samir Char · 2024
Learning representations of the world as a pretraining task before a supervised or reinforcement learning phase is a common application for self-supervised learning. In theory, if the learnt representations are good and broad enough, a new model based on them should require fewer instances to learn a specific job. It is generally expected that good self-supervised algorithms are also good few-shot learners. This research examines the few-shot capabilities of two well-known self-supervised learning algorithms for visual representations, SimCLR [1] and SimSiam [2], and compares them against a supervised counterpart. A common evaluation protocol is to train a linear classifier on top of (frozen) representations learnt by self-supervised methods. This protocol is taken a step further by evaluating supervised and self-supervised methods on a few-shot image classification task using frozen representations. The experiments find that, as expected, the supervised method has higher performance than self-supervised methods SimCLR [1] and SimSiam [2]. However, SimCLR [1] gives a consistently better performance and is proposed as a potentially good self-supervised approach to learn image representations for few-shot image tasks when supervised learning is not feasible. Furthermore, this opens up a new area of focus for future research - evaluating the efficacy of other SOTA self-supervised methods like BYOL [3] and Barlow Twins [4] for a variety of few-shot tasks across domains.