From COMET to COMES – Can Summary Evaluation Benefit from Translation Evaluation?
Mateusz Krubiński, Pavel Pecina · 2022
COMET is a recently proposed trainable neuralbased evaluation metric developed to assess the quality of Machine Translation systems.In this paper, we explore the usage of COMET for evaluating Text Summarization systems -despite being trained on multilingual MT outputs, it performs remarkably well in monolingual settings, when predicting summarization output quality.We introduce a variant of the model -COMES -trained on the annotated summarization outputs that uses MT data for pre-training.We examine its performance on several datasets with human judgments collected for different notions of summary quality, covering several domains and languages.