ChatPULI: Enhancement to the first Hungarian conversational model
Zijian Győző Yang, Ágnes Bánfi, Réka Dodé, Gergő Ferenczi, Flóra Földesi, Péter Hatvani, Enikö Héja, Mariann Lengyel, Gábor Madarász, Mátyás Osváth, Bence Sárossy, Kristóf Varga, Tamás Váradi, Gábor Prószéky, Noémi Ligeti-Nagy · Az Eszterházy Károly Tanárképző Főiskola tudományos közleményei. Tanulmányok a matematikai tudományok köréből/Az Eszterházy Károly Főiskola tudományos közleményei. Tanulmányok a matematikai tudományok köréből/Annales mathematicae et informaticae · 2025
This paper presents the development and evaluation of PULILlumiX- Llama-3.1 Chat and PULI Trio Q Chat, the first Hungarian-focused conversational large language models based on the Llama 3.1 and Qwen 2.5 architectures. Extending previous work on Hungarian instruction-following models, we applied continual pre-training on multilingual and Hungarian corpora, followed by supervised fine-tuning on an expanded instruction dataset including Hungarian, English, and Chinese prompts. Our models demonstrate significant performance improvements on Hungarian language understanding benchmarks, as well as on few-shot and zero-shot tasks, compared to earlier PULI models. Additionally, they show enhanced capabilities in machine translation and multi-turn dialogue handling. These results highlight the effectiveness of continual pre-training and fine-tuning strategies for adapting large language models to low-resource languages like Hungarian, and provide a foundation for future research in conversational AI for underrepresented languages.