Tutor-ICL: Guiding Large Language Models for Improved In-Context Learning Performance
Ikhyun Cho, Gaeul Kwon, Julia Hockenmaier · 2024
There has been a growing body of work focusing on the in-context learning (ICL) abilities of large language models (LLMs).However, it is an open question how effective ICL can be.This paper presents TUTOR-ICL, a simple prompting method for classification tasks inspired by how effective instructors might engage their students in learning a task.Specifically, we propose presenting exemplar answers in a comparative format rather than the traditional single-answer format.We also show that including the test instance before the exemplars can improve performance, making it easier for LLMs to focus on relevant exemplars.Lastly, we include a summarization step before attempting the test, following a common human practice.Experiments on various classification tasks, conducted across both decoderonly LLMs (Llama 2, 3) and encoder-decoder LLMs (Flan-T5-XL, XXL), show that TUTOR-ICL consistently boosts performance, achieving up to a 13.76% increase in accuracy.1