Majority Voting with Bidirectional Pre-translation For Bitext Retrieval
Alex Jones, Derry Tanti Wijaya · 2021
Obtaining high-quality parallel corpora is of paramount importance for training NMT systems.However, as many language pairs lack adequate gold-standard training data, a popular approach has been to mine so-called "pseudo-parallel" sentences from paired documents in two languages.In this paper, we outline some drawbacks with current methods that rely on an embedding similarity threshold, and propose a heuristic method in its place.Our method involves translating both halves of a paired corpus before mining, and then performing a majority vote on sentence pairs mined in three ways: after translating documents in language x → language y, after translating y → x, and using the original documents in languages x and y.We demonstrate success with this novel approach on the Tatoeba similarity search benchmark in 64 low-resource languages, and on NMT in Kazakh and Gujarati.We also uncover the effect of resource-related factors (i.e.how much monolingual/bilingual data is available for a given language) on the optimal choice of bitext mining method, demonstrating that there is currently no one-size-fits-all approach for this task.We make the code and data used in our experiments publicly available.1