Comparative Analysis of News Articles Summarization using LLMs
N Archanaa., B Shivanesh, Suwin Kumar. J.D. T, Bharathi Mohan G, Srinath Doss · 2024
In the rapidly evolving domain of Natural Language Processing (NLP), the efficiency of Large Language Models (LLMs) in generating abstractive text summaries plays a pivotal role in information synthesis. This study advances the understanding of LLM performance by conducting a comprehensive evaluation of seven cutting-edge models on four distinct datasets. The models selected for comparison include Distilbart-cnn-12-6, Led-base-16384, Google’s Bigbird, Microsoft’s ProphetNet, Facebook’s BART, T5 fine-tuned, and Google’s PEGASUS . Each model’s summarization prowess is rigorously assessed using a battery of metrics: ROUGE, METEOR, BERTScore, Cosine Similarity, and BLEU. The goal is to discern the intricate relationship between dataset characteristics and model efficacy, delivering insights into the inherent advantages and limitations of each model in handling specific data contexts. The results contribute to a refined understanding of LLM applicability, offering empirical evidence to aid in the selection of the most suitable model for varied summarization needs.