Linear regression based alignment of parallel texts using Homograph Words
António Ribeiro, Gabriel Pereira Lopes, João Tiago Mexia · European Conference on Artificial Intelligence · 2000
About 15% of the vocabulary found in large texts of the Official Journal of the European Communities is the same in its various official languages. If we take, for example, the Portuguese- Spanish pair, the rate rises to more than 30% since these are similar languages and, for the opposite reason, it drops to about 10% for the pair Portuguese-German. This is a wealthy source of information for parallel texts alignment that should not be left unused. Bearing this in mind, this paper describes a language independent method that makes use of those words, which are homograph for a pair of languages, in order to align parallel texts. This work was originally inspired and extends work done by Pascale Fung & Kathleen McKeown, and Melamed. In order to filter out words that may cause misalignment, we use confidence bands of linear regression analysis instead of statistically unsupported heuristics. We do not get 100% text alignment precision mostly due to term order policies in the different languages. The parallel segments obtained have an average length of four words for case law texts.