Evaluation of Neural Network Transformer Models for Named-Entity Recognition on Low-Resourced Languages
Ridewaan Hanslo · Annals of Computer Science and Information Systems · 2021
Neural Network (NN) models produce state-ofthe-art results for natural language processing tasks.Further, NN models are used for sequence tagging tasks on lowresourced languages with good results.However, the findings are not consistent for all low-resourced languages, and many of these languages have not been sufficiently evaluated.Therefore, in this paper, transformer NN models are used to evaluate named-entity recognition for ten low-resourced South African languages.Further, these transformer models are compared to other NN models and a Conditional Random Fields (CRF) Machine Learning (ML) model.The findings show that the transformer models have the highest F-scores with more than a 5% performance difference from the other models.However, the CRF ML model has the highest average F-score.The transformer model's greater parallelization allows lowresourced languages to be trained and tested with less effort and resource costs.This makes transformer models viable for low-resourced languages.Future research could improve upon these findings by implementing a linear-complexity recurrent transformer variant.