Automatic Classification of Scientific and Technical Papers Using Large Language Models and Retrieval-Augmented Generation
Jaehan Jeong, Dongsup Jin · Journal of information and communication convergence engineering · 2024
This study proposes artificial intelligence (AI) technology for the automatic classification of Korean scientific and technical papers, aiming to achieve high accuracy even with a small amount of labeled data.Unlike existing BERT-based Korean document classification models that perform supervised learning based on a large amount of accurately labeled data, this study proposes a structure that utilize large language models (LLMs) and retrieval-augmented generation (RAG) technology.The proposed method experimentally demonstrates that it can achieve higher accuracy than existing technologies across all cases using various amounts of labeled data.Furthermore, a qualitative comparison between manually-generated labels, and recognized as correct answers and those produced by LLM responses confirmed that the LLM responses were more accurate.The findings of this study, while limited to Korean scientific documents, provide evidence that a system utilizing LLM and RAG for document classification can easily be extended to other domains with diverse document datasets, owing to its effectiveness even with limited labels.