Development and Application of a Cross-language Document Comparability Metric
Fangzhong Su, Bogdan Babych · 2012
In this paper we present a metric that measures comparability of documents across different languages.The metric is developed within the FP7 ICT ACCURAT project, as a tool for aligning comparable corpora on the document level; further these aligned comparabl e documents are used for phrase alignment and extraction of translation equivalents, with the aim to extend phrase tables of statistica l MT systems without the need to use parallel texts.The metric uses several features, such as lexical information, document st ructure, keywords and named entities, which are combined in an ensemble manner.We present the results by measuring the reliability and effectiveness of the metric, and demonstrate its application and the impact for the task of parallel phrase extraction from comparable corpora.