Syntax and Semantics in Quality Estimation of Machine Translation
Rasoul Kaljahi, Jennifer Foster, Johann Roturier · 2014
We employ syntactic and semantic infor-mation in estimating the quality of ma-chine translation from a new data set which contains source text from English customer support forums and target text consisting of its machine translation into French. These translations have been both post-edited and evaluated by professional translators. We find that quality estima-tion using syntactic and semantic informa-tion on this data set can hardly improve over a baseline which uses only surface features. However, the performance can be improved when they are combined with such surface features. We also introduce a novel metric to measure translation ade-quacy based on predicate-argument struc-ture match using word alignments. While word alignments can be reliably used, the two main factors affecting the per-formance of all semantic-based methods seems to be the low quality of seman-tic role labelling (especially on ill-formed text) and the lack of nominal predicate an-notation. 1