Hitachi at SemEval-2020 Task 10: Emphasis Distribution Fusion on Fine-Tuned Language Models
Gaku Morio, Terufumi Morishita, Hiroaki Ozaki, Toshinori Miyoshi · 2020
This paper shows our system for SemEval-2020 task 10, Emphasis Selection for Written Text in Visual Media.Our strategy is two-fold.First, we propose fine-tuning many pre-trained language models, predicting an emphasis probability distribution over tokens.Then, we propose stacking a trainable distribution fusion (DISTFUSE) system to fuse the predictions of the fine-tuned models.Experimental results show that DISTFUSE is comparable or better when compared with a naive average ensemble.As a result, we were ranked 2nd amongst 31 teams.