Training Specialised Chatbots on Ukrainian Scientific Text Corpora Using Transfer Learning
Roman O. Liashenko, Сергій Олексійович Семеріков · 2024
This paper presents an extensive experimental study on applying transfer learning techniques to fine-tune pre-trained language models for creating chatbots specialised in Ukrainian scientific texts. The study involves curating two diverse text corpora from research publications, selecting appropriate base models, and fine-tuning them on the prepared datasets using the Hugging Face transformers library. The fine-tuned models are tested for generating chatbot responses to domain-specific user queries. Detailed analyses of the datasets, model architectures, fine-tuning process, and evaluation results are provided. The results demonstrate the feasibility and effectiveness of transfer learning for adapting language models to Ukrainian scientific texts and creating chatbots capable of meaningful dialogue in specialised domains. Challenges and future research directions are discussed.