AI-Driven Systematic Reviews: Transformer-Based Approaches to Text Classification
Ivan Terzic, Bjanka Vrljić, Danijel Mlinarić, Ana Meštrović, Ivica Botički · 2025
This paper explores the application of transformer-based models to automate the systematic literature review process, a traditionally labor-intensive task. Utilizing advanced NLP models from Hugging Face AutoTrain, a series of experiments were conducted on three manually annotated datasets: Mental Health (MH), Explainable Artificial Intelligence (EXAI), and Technology in Learning (LEARN). The experiments involved the fine-tuning of various transformer-based models and the exploration of ensemble approaches to enhance performance. The evaluation was performed using standard metrics, including accuracy, recall, and the F3 measure. Fine-tuned transformers demonstrated high accuracy and recall, particularly on balanced datasets, while ensemble models provided stable classification results by leveraging multiple transformers. Despite promising results, challenges such as high computational costs and the need for extensive hyperparameter tuning remain. This research demonstrates the potential of AI in supporting systematic literature reviews, reducing the workload while maintaining high accuracy.