JUSTers at SemEval-2020 Task 4: Evaluating Transformer Models against Commonsense Validation and Explanation
Ali Hussein Fadel, Mahmoud Al‐Ayyoub, Erik Cambria · 2020
In this paper, we describe our team's (JUSTers) effort in the Commonsense Validation and Explanation (ComVE) task, which is part of SemEval2020.We evaluate five pre-trained Transformer-based language models with various sizes against the three proposed subtasks.For the first two subtasks, the best accuracy levels achieved by our models are 92.90% and 92.30%, respectively, placing our team in the 12 th and 9 th places, respectively.As for the last subtask, our models reach 16.10 BLEU score and 1.94 human evaluation score placing our team in the 5 th and 3 rd places according to these two metrics, respectively.The latter is only 0.16 away from the 1 st place human evaluation score.