Cross-Lingual Similarity Discrimination with Translation Characteristics
Ying Hua Qin · International Journal of Artificial Intelligence & Applications · 2013
In cross-lingual plagiarism detection, the similarity between sentences is the basis of judgment.This paper proposes a discriminative model trained on bilingual corpus to divide a set of sentences in target language into two classes according their similarities to a given sentence in source language.Positive outputs of the discriminative model are then ranked according to the similarity probabilities.The translation candidates of the given sentence are finally selected from the top-n positive results.One of the problems in model building is the extremely imbalanced training data, in which positive samples are the translations of the target sentences, while negative samples or the non-translations are numerous or unknown.We train models on four kinds of sampling sets with same translation characteristics and compare their performances.Experiments on the open dataset of 1500 pairs of English Chinese sentences are evaluated by three metrics with satisfying performances, much higher than the baseline system.