Few-Shot Learning with Siamese Networks and Label Tuning
Thomas Müller, Guillermo Pérez-Torró, Marc Franco-Salvador · Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · 2022
We study the problem of building text classifiers with little or no training data, commonly known as zero and few-shot text classification.In recent years, an approach based on neural textual entailment models has been found to give strong results on a diverse range of tasks.In this work, we show that with proper pre-training, Siamese Networks that embed texts and labels offer a competitive alternative.These models allow for a large reduction in inference cost: constant in the number of labels rather than linear.Furthermore, we introduce label tuning, a simple and computationally efficient approach that allows to adapt the models in a few-shot setup by only changing the label embeddings.While giving lower performance than model fine-tuning, this approach has the architectural advantage that a single encoder can be shared by many different tasks.name task lang.train test labels token length