BLCU-ICALL at SemEval-2022 Task 1: Cross-Attention Multitasking Framework for Definition Modeling
Cunliang Kong, Yujie Wang, Ruining Chong, Liner Yang, Hengyuan Zhang, Erhong Yang, Yaping Huang · Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022) · 2022
This paper describes the BLCU-ICALL system used in the SemEval-2022 Task 1 Comparing Dictionaries and Word Embeddings, the Definition Modeling subtrack, achieving 1st on Italian, 2nd on Spanish and Russian, and 3rd on English and French.We propose a transformer-based multitasking framework to explore the task.The framework integrates multiple embedding architectures through the cross-attention mechanism, and captures the structure of glosses through a masking language model objective.Additionally, we also investigate a simple but effective model ensembling strategy to further improve the robustness.The evaluation results show the effectiveness of our solution.We release our code at: https://github.com/blcuicall/SemEval2022-Task1-DM.