LingJing at SemEval-2022 Task 1: Multi-task Self-supervised Pre-training for Multilingual Reverse Dictionary
Bin Li, Yixuan Weng, Fei Xia, Shizhu He, Bin Sun, Shutao Li · Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022) · 2022
This paper introduces the result of Team LingJing's experiments in SemEval-2022 Task 1 Comparing Dictionaries and Word Embeddings (CODWOE) 1 .This task aims at comparing two types of semantic descriptions, including the definition modeling and reverse dictionary track.Our team focuses on the reverse dictionary track and adopts the multi-task selfsupervised pre-training for multilingual reverse dictionaries.Specifically, the randomly initialized mDeBERTa-base model is used to perform multi-task pre-training on the multilingual training datasets.The pre-training step is divided into two stages, namely the MLM pretraining stage and the contrastive pre-training stage.As a result, all the experiments are performed on the pre-trained language model during fine-tuning.The experimental results show that the proposed method has achieved good performance in the reverse dictionary track, where we rank the 1-st in the Sgns targets of the EN and RU languages.