TFT-TL: Token-Level Filter Training Transfer Learning for Low-Resource Neural Machine Translation
Wei Dai, Dongfang Han, Turdi Tohti, Yi Liang, Zicheng Zuo, Yuanyuan Liao, Qingwen Yang · ACM Transactions on Asian and Low-Resource Language Information Processing · 2025
Transfer learning plays a crucial role in low-resource machine translation by addressing the challenge of poor model performance due to limited data in low-resource languages, thereby improving translation accuracy. Current research methods not only utilize pre-trained parent models for parameter initialization and fine-tuning but also use the soft labels output by these parent models to enhance the consistency between parent and child models. However, even if the parent model performs well, there are still instances where certain token predictions are unstable. During training, if the child model incorporates these unstable token predictions, it can hinder its learning effectiveness; the child model might not fully comprehend the parent model’s prediction strategy, potentially affecting overall translation performance. To address this, we propose a training strategy called Token-Level Filter Training, designed to effectively filter out unstable token predictions from the parent model, thereby transferring the parent model’s positive knowledge to the child model. Additionally, we introduce a hierarchical ranking loss method to help the child model better learn the parent model’s prediction strategies and sequence order, thus enhancing translation accuracy and fluency. Experimental results show that our method outperforms baseline methods on the public datasets Global Voices (Id, Ca, Hu, Pl) and WMT17 (Turkish–English), with BLEU score improvements of 1.47, 0.91, 0.50, 0.54, and 0.55, respectively. These results demonstrate the effectiveness and superiority of the proposed method.