TransPrompt: Towards an Automatic Transferable Prompting Framework for Few-shot Text Classification
Chengyu Wang, Jianing Wang, Minghui Qiu, Jun Huang, Ming Zhou Gao · Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing · 2021
Recent studies have shown that prompts improve the performance of large pre-trained language models for few-shot text classification.Yet, it is unclear how the prompting knowledge can be transferred across similar NLP tasks for the purpose of mutual reinforcement.Based on continuous prompt embeddings, we propose TransPrompt, a transferable prompting framework for few-shot learning across similar tasks.In TransPrompt, we employ a multitask meta-knowledge acquisition procedure to train a meta-learner that captures cross-task transferable knowledge.Two de-biasing techniques are further designed to make it more task-agnostic and unbiased towards any tasks.After that, the meta-learner can be adapted to target tasks with high accuracy.Extensive experiments show that TransPrompt outperforms single-task and cross-task strong baselines over multiple NLP tasks and datasets.We further show that the meta-learner can effectively improve the performance on previously unseen tasks.TransPrompt also outperforms strong fine-tuning baselines when learning with full training sets.