Instruction Tuning for Developing Large Language Models Specialized in Chemical Domain
Jung‐Min Lee, H.Y. Kim, Sungsu Lee, Yunsoo Kim, K.S. Lee, Seoung-Bum Kim · Journal of Korean Institute of Industrial Engineers · 2025
In recent years, notable progress in natural language processing (NLP) has been attributed to the advent of large language models (LLM), exemplified by innovations such as chat generative pre-trained transformer (ChatGPT) developed by OpenAI. Complementary LLMs such as Llama and Vicuna, are also being rapidly developed. Despite these advancements, LLMs still have limitations because of their training on generic text sources like Wikipedia. While some LLMs are trained on data spanning multiple domains, they often exhibit suboptimal performance in specialized domains like chemistry or finance. This study aims to address this limitation by developing an LLM specialized for the chemical domain through instruction tuning. We use a diverse dataset in both Korean and English, encompassing multiple tasks such as summarization, keyword extraction, and question answering for instruction tuning. The baseline model for our study is Llama2. Evaluation using both Korean and English datasets demonstrates that our specialized Llama2 outperforms its untuned counterpart in both languages. This demonstrates the effectiveness of our approach in enhancing the performance of models across languages and domains, particularly in chemistry. Additionally, we propose efficient instruction tuning strategies through various experiments.