Evaluating the Quality of Web-Mined Bilingual Sentences Using Multiple Linguistic Features
Xiaohua Liu, Ming Zhou · 2010
We raise the problem of evaluating the quality of bilingual sentences mined from the web, which is critical for such applications as statistical machine translation (SMT) and English as Second Language (ESL) learning. To tackle this problem, we propose a novel method that integrates multiple linguistic features related to spelling, grammar, and alignment, particularly the sentence type feature that indicates if a sentence can be parsed by the Link Grammar Parser (LGP). Promising results are achieved on a bilingual corpus of about 6 million English-Chinese sentences mined from the web, indicating the effectiveness of our proposed method.