Quick and Reliable Document Alignment via TF/IDF-weighted Cosine Distance
Christian Buck, Philipp Koehn · 2016
This work describes our submission to the WMT16 Bilingual Document Alignment task.We show that a very simple distance metric, namely Cosine distance of tf/idf weighted document vectors provides a quick and reliable way to align documents.We compare many possible variants for constructing the document vectors.We also introduce a greedy algorithm that runs quicker and performs better in practice than the optimal solution to bipartite graph matching.Our approach shows competitive performance and can be improved even further through combination with URL based pair matching.