Increase the efficiency of English-Chinese sentence alignment: target range restriction and empirical selection of stop words
Wing‐Kwong Wong, Hsi-Hsun Yang, Wei-Lung Shen, Sheng-Kai Yin, Sheng-Cheng Hsu · 2008
Abstract:- In this paper, we use a lexical method to do sentence alignment for an English-Chinese corpus. Past research shows that alignment using a dictionary involves a lot of word matching and dictionary look ups. To address these two issues, we first restrict the range of candidate target sentences, based on the location of the source sentence relative to the beginning of the text. Moreover, careful empirical selection of stop words, based on word frequencies in the source text, helps to reduce the number of dictionary look ups. Experimental results show that the amount of word matching can be cut down by 75 % and that of dictionary look ups by as much as 43 % without sacrificing precision and recall. Another experiment was also done with twenty New York Times articles with 598 sentences and 18395 words. The resulted precision is 95.6 % and the recall is 93.8%. Among all predicted alignment, 86 % of the alignment is 1:1 (one source sentence to one target sentence), 8 % is 1:2, and 6 % is 2:1. Further analysis shows that most errors occur in alignments of types 1:2 and 2:1. Future work should focus on problems with these two alignment types. Key-Words:- Sentence alignment, lexical method, statistical method, English-Chinese corpus, stop words,