Towards a Predicate-Argument Evaluation for MT
Ondřej Bojar, Dekai Wu · Institute of Formal and Applied Linguistics (ÚFAL) · 2012
HMEANT (Lo and Wu, 2011a) is a manual MT evaluation technique that focuses on predicate-argument structure of the sentence. We relate HMEANT to an established lin- guistic theory, highlighting the possibilities of reusing existing knowledge and resources for interpreting and automating HMEANT. We apply HMEANT to a new language, Czech in particular, by evaluating a set of English- to-Czech MT systems. HMEANT proves to correlate with manual rankings at the sentence level better than a range of automatic met- rics. However, the main contribution of this paper is the identification of several issues of HMEANT annotation and our proposal on how to resolve them.