Chinese--Japanese Parallel Sentence Extraction from Quasi--Comparable Corpora
Chenhui Chu, Toshiaki Nakazawa, Sadao Kurohashi · 2013
Parallel sentences are crucial for statistical machine translation (SMT). However, they are quite scarce for most language pairs, such as Chinese–Japanese. Many studies have been conducted on extracting parallel sentences from noisy parallel or compara-ble corpora. We extract Chinese–Japanese parallel sentences from quasi–comparable corpora, which are available in far larger quantities. The task is significantly more difficult than the extraction from noisy parallel or comparable corpora. We ex-tend a previous study that treats parallel sentence identification as a binary classifi-cation problem. Previous method of clas-sifier training by the Cartesian product is not practical, because it differs from the real process of parallel sentence extrac-tion. We propose a novel classifier train-ing method that simulates the real sentence extraction process. Furthermore, we use linguistic knowledge of Chinese character features. Experimental results on quasi– comparable corpora indicate that our pro-posed approach performs significantly bet-ter than the previous study.