Fine-Tuning LLaMA 3.2-1B for Long-Text Summarization: A Case Study on Book Summarization
Rareş-George Diaconu, Ștefan-Daniel Achirei · 2025
In the era of information overload, effective text summarization has emerged as a critical area within natural language processing (NLP). This paper presents a novel approach to automatic text summarization by fine-tuning the Meta-LLama/Llama-3.2-1B model, leveraging two important datasets: DailyMail/CNN, which focuses on narrative summarization, and BigPatent, a domain-specific dataset for technical document summarization. Our methodology involves training the model to generate concise and coherent summaries while evaluating its performance using established metrics such as ROUGE scores. The results demonstrate moderate improvements in both accuracy and processing speed, highlighting the model's capability to handle diverse text types. Additionally, we analyze the impact of dataset characteristics on summarization quality, providing insights for future research in task-specific fine-tuning. This work contributes to the ongoing development of efficient and effective summarization systems.