PromptDA: Label-guided Data Augmentation for Prompt-based Few Shot Learners
Canyu Chen, Kai Shu · 2023
Recent advances in large pre-trained language models (PLMs) lead to impressive gains on natural language understanding (NLU) tasks with task-specific fine-tuning.However, directly fine-tuning PLMs heavily relies on sufficient labeled training instances, which are usually hard to obtain.Prompt-based tuning on PLMs has shown to be powerful for various downstream few-shot tasks.Existing works studying prompt-based tuning for few-shot NLU tasks mainly focus on deriving proper label words with a verbalizer or generating prompt templates to elicit semantics from PLMs.In addition, conventional data augmentation strategies such as synonym substitution are also widely adopted in low-resource scenarios.However, the improvements they bring to prompt-based few-shot learning have been demonstrated to be marginal.Thus, an important research question arises as follows: how to design effective data augmentation methods for prompt-based fewshot tuning?To this end, considering the label semantics are essential in prompt-based tuning, we propose a novel label-guided data augmentation framework PROMPTDA, which exploits the enriched label semantic information for data augmentation.Extensive experiment results on few-shot text classification tasks show that our proposed framework achieves superior performances by effectively leveraging label semantics and data augmentation for natural language understanding.