PILLOW: Enhancing Efficient Instruction Fine-tuning via Prompt Matching
Zhenting Qi, Xiaoyu Tan, Shaojie Shi, Chao Yun Qu, Yinghui Xu, Qi Yuan · 2023
Instruction fine-tuning has conventionally been employed to adapt Large Language Models (LLMs) to a variety of tasks.Nonetheless, this technique often necessitates substantial computational resources, making it impractical for deployment by individuals or small-scale entities.Recently, Low-Rank Adaptation (LoRA) has become a promising alternative, offering high capabilities on par with full tuning with reduced resource overhead.However, attaining satisfactory performance through the fine-tuning of LoRA is a non-trivial challenge.In this paper, we propose PILLOW, which aims to improve LoRA's performance by a discriminationbased prompting method, leveraging LLMs' In-Context Learning ability.PILLOW incorporates a matching network that selects prompts from a user-defined prompt pool, concatenates the selected prompts with the user instruction as input, and performs inference using the LoRAfine-tuned LLMs.Trained with Reinforcement Learning, PILLOW exhibits commensurate performance on various evaluation metrics compared with typical instruction fine-tuning methods, utilizing only consumer-grade GPU resources and exhibiting a large reduction in computational costs.