A New Combined Lexical and Statistical based Sentence Level Alignment Algorithm for Parallel Texts.
Grigori Sidorov, Juan Pablo Posadas-Durán, Héctor Jiménez-Salazar, Liliana Chanona-Hernández · 2011
Parallel texts alignment is an active research area in Natural Language Processing field. In this paper, we propose a method for sentence alignment of parallel texts that is based both on lexical and statistical information. The alignment procedure uses dynamic programming technique. We made our experiments for Spanish and English texts. We use lexical information from bilingual Spanish-English dictionary, as well as the sentence length measured in words and in characters. The proposed method was tested on a corpus of fiction texts, where the frequency of multiple alignments, omissions and insertions is higher than in other types of texts. We obtained better results than the standard Vanilla aligner system that uses a purely statistical approach. KEY WORDS: Parallel texts alignment, English, Spanish, dynamic programming, lexical-based alignment, statistical alignment, anchor points, sentence length, fiction writing. 1