Contrastive Learning for Universal Zero-Shot NLI with Cross-Lingual Sentence Embeddings
Md. Kowsher, Md. Shohanur Islam Sobuj, Nusrat Jahan Prottasha, Mohammad Shamsul Arefin, Yasuhiko Morimoto · 2023
Natural Language Inference (NLI) is a crucial task in natural language processing, involving the classification of sentence pairs into entailment, contradiction, or neutral categories.This paper introduces a novel approach to achieve universal zero-shot NLI by employing contrastive learning with cross-lingual sentence embeddings.We utilize a large-scale pretrained multilingual language model trained on NLI data from 15 diverse languages, enabling our approach to achieve zero-shot performance across other unseen languages during the training, including low-resource ones.Our method incorporates a Siamese network-based contrastive learning framework to establish semantic relationships among similar sentences in the 15 languages.By training the zero-shot NLI model using contrastive training on this multilingual data, it effectively captures meaningful semantic relationships.Leveraging the fine-tuned language model's zero-shot learning capabilities, our approach extends the zeroshot capability to additional languages within the multilingual model.Experimental results demonstrate the effectiveness of our approach in achieving universal zero-shot NLI across diverse languages, including those with limited resources.We showcase our method's ability to handle previously unseen low-resource language data within the multilingual model, highlighting its practical applicability and broad language coverage.Encoder