Refining Abstractive Text Summarization: Towards an Optimizing Approach based on PTLM Selection and Fine-Tuning
Adebayo Kabeer, Samiya Khan · 2024
This study provides an out-of-the-box inferential and empirical analysis of four state-of-the-art large natural language models namely, BART, PEGASUS, ProphetNet, and T5, assessing their summarization capabilities on three diverse datasets, CNN/DailyMail, Gigaword, and XSUM. The analysis uniquely positions itself by considering the models’ performance without any task-specific fine-tuning, the results are quantified using the ROUGE metric, offering insights into each model’s ability to generate coherent and concise summaries directly out-of-the-box. The results indicated significant variations in performance, with BART and PEGASUS generally leading in terms of ROUGE scores and actual summary inspection, T5 excelling in generating concise summaries with limited information and mainly being extractive (copying), and ProphetNet showing a need for potential task-specific adjustments. The findings offer guidance for selecting and deploying pre-trained language models for real-world summarization applications within a low-resource environment.