APG: Automatic Prompt Generation for Improved Document Summarization
Jacob Parnell, Iñigo Jauregi Unanue, Scott L. Matthews, Massimo Piccardi · IEEE Transactions on Audio Speech and Language Processing · 2025
In recent years, prompting has become a mainstream technique for querying language models in natural language generation tasks such as machine translation, question answering and document summarization. In document summarization in particular, prompts have been used to control aspects of the predicted summary (e.g., style and length) and generally improve its quality. However, all the prompt-based summarization approaches proposed to date rely on significant manual design effort or annotation of additional training data. For this reason, in this paper we propose a novel approach for document summarization – nicknamed automatic prompt generation (APG) – allowing the model to learn to simultaneously generate its own prompts and the predicted summaries without the need for any supplementary annotations. This capability has been achieved by augmenting the training data with sets of keywords automatically extracted from the input documents with an off-the-shelf, unsupervised keyword extractor, and generating the prompts directly from the hidden states of the model's encoder. The proposed approach has been evaluated over five, diverse summarization datasets, showing that its performance has proved higher than that of many baseline models, including a state-of-the-art large language model, with increases of up to$+9.50$ROUGE$R_{1}$pp over BART-base, and$+4.02$$F_{BERT}$score pp over GPT-4o mini, as well as evidence of improved generalization.