Weakly Augmented Suffix‐Attention Adapters for Few‐Shot POS Tagging on Pretrained LLMs
Alim Murat, Qi Yuan, Samat Ali · International Journal of Intelligent Systems · 2025
Part‐of‐speech (POS) tagging in agglutinative, low‐resource languages suffers from data sparsity and out‐of‐vocabulary (OOV) issues due to rich affixal morphology. We propose a parameter‐efficient suffix‐aware attention (SAA) framework that (i) explicitly models stem–suffix interactions via per‐layer suffix‐attention blocks, (ii) integrates these modules into a frozen pretrained transformer backbone through lightweight adapters, and (iii) augments few‐shot training data with weakly supervised suffix recombination to double effective examples. We evaluate our approach on three languages including Uyghur, Uzbek, and Kyrgyz under k‐shot setting, comparing against strong baselines including full fine‐tuning, adapter‐only tuning, and character‐level taggers. Our model consistently achieves the highest overall F 1 (up to 81.5% on Uyghur), OOV F 1 (over 63%), and suffix recall (nearly 70%) across all settings, yielding average gains of 4‐5 points over Adapter‐Only baselines. Ablations confirm that SAA is the primary driver of improvements, while augmentation and KL regularization further stabilize learning. Error and noise‐robustness analyses demonstrate that explicit morphological attention effectively mitigates segmentation errors and reduces key tagging failures under extreme low‐resource conditions. These results validate the efficacy of combining morphological inductive bias with parameter‐efficient fine‐tuning for robust POS tagging in morphologically rich, low‐resource languages.