Improving Part-of-Speech Tagging with Relative Positional Encoding in Transformer Models and Basic Rules
Jerome Aondongu Achir, Abdukarim Mohammad, Mohammed Abdullahi · Indonesian Journal of Data and Science · 2025
Part-of-speech (POS) tagging is a critical task in natural language processing (NLP), influencing the performance of downstream applications such as semantic parsing and machine translation. This study presents a novel approach to POS tagging by incorporating relative positional encoding within the transformer model. Unlike traditional absolute positional encoding, the proposed method leverages token dependencies more effectively, enhancing the transformer’s self-attention mechanism. The architecture integrates a rule-based module to correct misclassifications, further refining the results. Experiments on the Groningen Meaning Bank (GMB) dataset demonstrate the model's superiority, achieving an accuracy of 99.68%, a significant improvement over the 98.60% accuracy of models with absolute positional encoding. Additional metrics, including precision (0.92), recall (0.89), and F1-score (0.90), further confirm the model's effectiveness. The findings highlight the potential of relative positional encoding in improving contextual understanding and model performance, providing a robust solution for POS tagging tasks in NLP.