Parallel Sentence Mining by Constrained Decoding
Pinzhen Chen, Nikolay Bogoychev, Kenneth Heafield, Faheem Kirefu · 2020
We present a novel method to extract parallel sentences from two monolingual corpora, using neural machine translation.Our method relies on translating sentences in one corpus, but constraining the decoding by a prefix tree built on the other corpus.We argue that a neural machine translation system by itself can be a sentence similarity scorer and it efficiently approximates pairwise comparison with a modified beam search.When benchmarked on the BUCC shared task, our method achieves results comparable to other submissions.