Identifying Common Challenges for Human and Machine Translation: A Case Study from the GALE Program
Lauren Friedman, Stephanie M. Strassel · 2008
1-0003. The content of this paper does not necessarily reflect the position or the policy of the Government, and no official en-dorsement should be inferred. The dramatic improvements shown by statisti-cal machine translation systems in recent years clearly demonstrate the benefits of hav-ing large quantities of manually translated parallel text for system training and develop-ment. And while many competing evaluation metrics exist to evaluate MT technology, most of those methods also crucially rely on the ex-istence of one or more high quality human translations to benchmark system perform-ance. Given the importance of human transla-tions in this framework, understanding the particular challenges of human translation-for-MT is key, as is comprehending the relative strengths and weaknesses of human versus machine translators in the context of an MT evaluation. Vanni (2000) argued that the met-ric used for evaluation of competence in hu-man language learners may be applicable to MT evaluation; we apply similar thinking to improve the prediction of MT performance, which is currently unreliable. In the current paper we explore an alternate model based upon a set of genre-defining features that prove to be consistently challenging for both humans and MT systems. 1