Pre-trained Language Models Can be Fully Zero-Shot Learners
Xuandong Zhao, Siqi Ouyang, Zhiguo Yu, Ming Wu, Lei Li · 2023
How can we extend a pre-trained model to many language understanding tasks, without labeled or additional unlabeled data?Pre-trained language models (PLMs) have been effective for a wide range of NLP tasks.However, existing approaches either require fine-tuning on downstream labeled datasets or manually constructing proper prompts.In this paper, we propose nonparametric prompting PLM (NPPrompt) for fully zero-shot language understanding.Unlike previous methods, NPPrompt uses only pre-trained language models and does not require any labeled data or additional raw corpus for further fine-tuning, nor does it rely on humans to construct a comprehensive set of prompt label words.We evaluate NPPrompt against previous major fewshot and zero-shot learning methods on diverse NLP tasks: text classification, text entailment, similar text retrieval, paraphrasing, and multiple-choice question answering.Experimental results demonstrate that our NPPrompt outperforms the previous best fully zero-shot method by big margins, with absolute gains of 12.8% in accuracy on text classification and 15.6% on the GLUE benchmark.