Quality Estimation Using Minimum Bayes Risk
Subhajit Naskar, Daniel Deutsch, Markus Freitag · 2023
This report describes the Minimum Bayes Risk Quality Estimation (MBR-QE) submission to the Workshop on Machine Translation's 2023 Metrics Shared Task.MBR decoding with neural utility metrics like BLEURT is known to be effective in generating high quality machine translations.We use the underlying technique of MBR decoding and develop an MBR based reference-free (quality estimation) metric.Our method uses an evaluator machine translation system and a reference-based utility metric (specifically BLEURT and Met-ricX) to calculate a quality estimation score of a model's output.We report results related to comparing different MBR configurations and utility metrics.