Evaluating Metrics for Document-context Evaluation in Machine Translation
Vikas Raunak, Tom Kocmi, Matt Post · 2023
We describe our submission of a new metric, SLIDE (Raunak et al., 2023), to the WMT 2023 metrics task.SLIDE is a reference-free qualityestimation metric that works by constructing a fixed sentence-length window over the documents in a test set, concatenating chunks and then sending them for scoring as a single unit by COMET (Rei et al., 2022a).We find that SLIDE improves dramatically over its contextless counterpart on the two WMT22 evaluation campaigns (MQM and DA+SQM).