Cross-lingual Sentence Embedding using Multi-Task Learning
Koustava Goswami, Sourav Dutta, Haytham Assem, Theodorus Fransen, John Philip McCrae · Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing · 2021
Multilingual sentence embeddings capture rich semantic information not only for measuring similarity between texts but also for catering to a broad range of downstream crosslingual NLP tasks.State-of-the-art multilingual sentence embedding models require large parallel corpora to learn efficiently, which confines the scope of these models.In this paper, we propose a novel sentence embedding framework based on an unsupervised loss function for generating effective multilingual sentence embeddings, eliminating the need for parallel corpora.We capture semantic similarity and relatedness between sentences using a multitask loss function for training a dual encoder model mapping different languages onto the same vector space.We demonstrate the efficacy of an unsupervised as well as a weakly supervised variant of our framework on STS, BUCC and Tatoeba benchmark tasks.The proposed unsupervised sentence embedding framework outperforms even supervised stateof-the-art methods for certain under-resourced languages on the Tatoeba dataset and on a monolingual benchmark.Further, we show enhanced zero-shot learning capabilities for more than 30 languages, with the model being trained on only 13 languages.Our model can be extended to a wide range of languages from any language family, as it overcomes the requirement of parallel corpora for training.