Neural Network and Transformer-Based PoS Tagger for Low Resource Languages
Endrit Fetahi, Mentor Hamiti, Arsim Susuri, Besnik Selimi, Deshira Imeri Saiti · 2024
NLP (Natural Language Processing) is a wide area of research nowadays since many different technologies are involved in order for machines to understand and process information. This field presents particular challenges regarding low-resource languages, as there is a low amount of research done. This shows a significant gap in relation to Computational Linguistics in these languages. Part-of-Speech (PoS) tagging is a key focus in low-resource languages as it serves as a foundation for further investigation. In this paper, we present a PoS Tagger for the Albanian language as an experimental language, utilizing deep learning models such as RNN, LSTM, Bi-LSTM, GRU, and Transformers. The experiments were conducted in three setups, and the accuracy is measured through various metrics. Our findings reveal that the Fine-Tuned Transformers architecture delivers the most favorable results, achieving an F1 score of 95%.