Unsupervised Machine Translation Based on Dynamic Adaptive Masking Strategy and Multi-Task Learning
Chuancai Zhang, Dan Qu, Liming Du, Kaiyuan Yang · 2024
This study proposes an unsupervised machine translation method based on a dynamic adaptive masking strategy and multi-task learning. Firstly, a dynamic adaptive masking strategy is introduced in masked language modeling, dynamically adjusting the masking rate based on sentence complexity and contextual information to retain more important information in complex sentences while optimizing the masking effect. Secondly, a multi-task learning framework is adopted, incorporating tasks such as translation, summarization, and question answering into the unsupervised translation model. By using a shared encoder and task-specific decoders for joint training, the model enhances both generalization and task-specific capabilities. Experimental results demonstrate that the proposed approach significantly improves translation quality and model generalization across multiple translation tasks, providing new insights and application prospects for unsupervised machine translation research.