Error Analysis of Verb Inflections in Spanish Translation Output
Maja Popović, Hermann Ney · RWTH Publications (RWTH Aachen) · 2006
Evaluation of machine translation output is an important but difficult task. Over the last years, various automatic evaluation measures have been studied and have become widely used. However, these measures do not give any details about the nature of translation errors. Therefore some analysis of the generated output is needed in order to identify the main problems and possibilities for improvements. In this work, we present the results of automatic error analysis of Spanish verbs in statistical machine translation output generated by RWTH in the second TC-STAR evaluation. Different types of verb inflections referring to the mode, tense and person are defined. For each inflection type, PER-based precision and recall measures are calculated as well as corresponding F-measure. Additionally, a ratio between relative frequency and PER-based F-measure is defined in order to estimate significance of each verb inflection. Analysis based on the F-measure and the relative frequency has shown which verb inflections are the most difficult to translate and which are the most important to improve. Analysis of the PER-based precision-recall graph has indicated which inflections are tending to be translated wrongly and which are tending to replace other inflections. 1. Introduction and Related