Bridging the Gap using Contrastive Learning and Semantic Consistency for Unsupervised Neural Machine Translation
Chuancai Zhang, Dan Qu, Liming Du, Kaiyuan Yang · 2024
In Unsupervised Neural Machine Translation (UNMT) tasks, the lack of extensive parallel corpora makes it challenging for the model to directly optimize the correspondence between the source and target languages. UNMT models primarily rely on learning from monolingual data, making it difficult to fully capture cross-linguistic semantic alignment and consistency during training. Therefore, effectively enhancing semantic alignment between the source and target languages under unsupervised conditions has become a key challenge in the field of UNMT. To address this issue, this study proposes a semantic alignment and consistency enhancement mechanism based on contrastive learning, aiming to improve the model's ability to capture cross-lingual semantic differences by constructing an optimization objective focused on semantic consistency. The method utilizes a pretrained cross-lingual model to generate semantic embeddings for both the source and target languages, then designs a contrastive learning-based loss function that minimizes the semantic distance between correct translations and source sentences while increasing the semantic distance between incorrect translations and source sentences, thereby enhancing the model's performance in semantic consistency. The experiments validate the effectiveness and broad applicability of the semantic consistency optimization method based on contrastive learning, demonstrating great potential and advantages, especially in multilingual tasks of UNMT. The model achieved significant improvements in BLEU scores across various language pairs.