Entailment as Robust Self-Learner
Jiaxin Ge, Hongyin Luo, Yoon Kim, James Glass · 2023
Entailment has been recognized as an important metric for evaluating natural language understanding (NLU) models, and recent studies have found that entailment pretraining benefits weakly supervised fine-tuning.In this work, we design a prompting strategy that formulates a number of different NLU tasks as contextual entailment.This approach improves the zero-shot adaptation of pretrained entailment models.Secondly, we notice that self-training entailment-based models with unlabeled data can significantly improve the adaptation performance on downstream tasks.To achieve more stable improvement, we propose the Simple Pseudo-Label Editing (SimPLE) algorithm for better pseudo-labeling quality in self-training.We also found that both pretrained entailmentbased models and the self-trained models are robust against adversarial evaluation data.Experiments on binary and multi-class classification tasks show that SimPLE leads to more robust self-training results, indicating that the self-trained entailment models are more efficient and trustworthy than large language models on language understanding tasks.