Adapting Large Language Models to Vietnamese Law: Pretrained LLM Refinement vs Retrieval Augmented Generation
Nguyen Nguyen, Thang V.Q. Le, Anh-Cuong Le, Viet-Ha Nguyen, Viet-Cuong Nguyen · 2024
Large Language Models (LLMs) have shown their potential in a wide range of tasks, but much of their development has been concentrated on the English language. This focus has created a noticeable gap in the availability of LLMs for other languages, as well as in specialized domains such as legal contexts. In this study, we aim to address this gap by developing a Vietnamese Large Language Model and building a Retrieval-Augmented Generation (RAG) system within legal settings. Our methodology includes building and processing legal datasets, followed by training the LLM specifically for Vietnamese legal applications. Our findings suggest that both the LLM and the RAG system perform well in retaining legal knowledge and providing more reliable answers to legal inquiries. These results highlight the potential of these approaches for legal tasks in the Vietnamese language. This research not only contributes to the application of AI in the legal field but also offers important insights into the development of AI solutions for non-English languages, addressing a critical gap in current AI research and supporting more inclusive language model applications.