In-context Learning of Large Language Models for Controlled Dialogue Summarization: A Holistic Benchmark and Empirical Analysis
Yuting Tang, Ratish Puduppully, Zhengyuan Liu, Nancy F. Chen · 2023
Large Language Models (LLMs) have shown significant performance in numerous NLP tasks, including summarization and controlled text generation.A notable capability of LLMs is in-context learning (ICL), where the model learns new tasks using input-output pairs in the prompt without any parameter update.However, the performance of LLMs in the context of few-shot abstractive dialogue summarization remains underexplored.This study evaluates various state-of-the-art LLMs on the SAMSum dataset within a few-shot framework.We assess these models in both controlled (entity control, length control, and person-focused planning) and uncontrolled settings, establishing a comprehensive benchmark in few-shot dialogue summarization.Our findings provide insights into summary quality and model controllability, offering a crucial reference for future research in dialogue summarization.