Self-Adaptive In-Context Learning: An Information Compression Perspective for In-Context Example Selection and Ordering
Zhiyong Wu, Yaoxiang Wang, Jiacheng Ye, Lingpeng Kong · 2023
Despite the impressive few-shot performance of in-context learning (ICL), it remains a common practice to randomly select examples to serve as the context.In this paper, we advocate self-adaptive in-context learning, a new principle for ICL, in which the self-adaption mechanism is introduced to help each input find an in-context example organization (i.e., selection and permutation) that can derive the correct output, thus maximizing performance.To validate the effectiveness of self-adaptive ICL, we propose a general select-then-rank framework and a set of novel selection and ranking algorithms.Upon extensive evaluation on eight different NLP datasets, our self-adaptive ICL method achieves a 40% relative improvement over the common practice setting.Further analysis reveals the great potential of selfadaptive ICL as a promising method to close the gap between ICL and finetuning.Our code will be released to facilitate future research.