ON-LINE COMPILATION OF COMPARABLE CORPORA AND THEIR EVALUATION
Radu Ion, Dan Ioan Tufis, Tiberiu Boroş, Alexandru Ceauşu, Dan C. Stefanescu · 2010
Using comparable corpora is became a topic in the mainstream Machine Translation (MT) research because, for less resourced languages, mining the Web for comparable corpora is assumed to be more productive than searching for parallel corpora. The experiments in using comparable corpora in enhancing translation models demonstrated significant improvements in MT accuracy. This paper reports on specific procedures of building comparable corpora from Wikipedia and from general Web using a highly customizable application that can merge diverse web crawlers and source their output either into files or NLP web services. We also describe a method of scoring a pair of documents from a comparable corpus as to their parallelism degree.