A Fine-Tuned BART Pre-trained Language Model for the Indonesian Question-Answering Task
Alfonso Darren Vincentio, Seng Hansun · Engineering Technology & Applied Science Research · 2025
The information extraction process from a given context can be time consuming and a Pre-trained Language Model (PLM) based on the transformer architecture could reduce the time needed to obtain the information. Moreover, PLM is easily fine-tuned to accomplish certain tasks, one of which is the Question-Answering (QA) task. In literature, QA tasks are generally fine-tuned using encoder-based PLMs, such as the Bidirectional Encoder Representations from Transformers (BERT), where the generated answers come from the extraction process of the context. In order to be able to return more abstract answers, a PLM with Natural Language Generation (NLG) capability, such as the Bidirectional and Auto-Regressive Transformer (BART), is needed. In this study, we aim to fine-tune the NLG PLM using BART to build a more abstractive generative QA task. Based on the experimental results, the fine-tuned BART model performs well with an 85.84 F1 score and a 59.42 Exact Match (EM) score.