Comparison of Fine-tuning vs. RAG in sLLM - Focusing on Machine Reading Comprehension and Sentiment Analysis
G. Kim, Do-Guk Kim · Korean Institute of Smart Media · 2025
Studies have actively compared fine-tuning and retrieval-augmented generation (RAG) to enhance the adaptability of large language models (LLMs) to specific domains. Although question and source document pairs are already available in certain environments, no comparative study has assessed whether RAG has the potential to improve domain-specific performance more easily than fine-tuning in such settings. In this study, we compare the performance of fine-tuning and RAG for Korean machine reading comprehension (KorQuAD) and sentiment analysis (NSMC) tasks using small language models (sLLM). Our results show that RAG improved performance by 10.2% on KorQuAD and 32.3% on NSMC. when RAG and fine-tuning were combined, performance improved by 11.5% and 41.9%, respectively. Our results indicate that RAG is advantageous when resources are limited, while a complementary use of both methods can outperform fine-tuning when sufficient resources are available.