Recovering document annotations for sentence-level bitext
Rachel Wicks, Matt Post, Philipp Koehn · 2024
Data availability limits the scope of any given task.In machine translation, historical models were incapable of handling longer contexts, so the lack of document-level datasets was less noticeable.Now, despite the emergence of long-sequence methods, we remain within a sentence-level paradigm and without data to adequately approach context-aware machine translation.Most large-scale datasets have been processed through a pipeline that discards document-level metadata.In this work, we reconstruct document-level information for three (ParaCrawl, News Commentary, and Europarl) large datasets in German, French, Spanish, Italian, Polish, and Portuguese (paired with English).We then introduce a document-level filtering technique as an alternative to traditional bitext filtering.We present this filtering with analysis to show that this method prefers context-consistent translations rather than those that may have been sentence-level machine translated.Last we train models on these longer contexts and demonstrate improvement in document-level translation without degradation of sentence-level translation.We release our dataset, PARADOCS, and resulting models as a resource to the community.1Art is trying to become science. Die Kunst versucht, Wissenschaft zu sein.It deals with philosophy, sociology and psychology, and desires to be a pedagogical resource too.It wants to be political, yet maintain the charisma of the underground (that is, by being non-political).It wants to be a commodity.Sie beschäftigt sich mit Philosophie, Soziologie, Psychologie und möchte eine Quelle für die Pädagogik sein.Die Kunst will politisch sein, aber sich das Charisma des unpolitischen Undergrounds erhalten.Sie will Ware sein.…. Culture without qualification is fraud. Art is trying to become science.It deals with philosophy, sociology and psychology, and desires to be a pedagogical resource too.It wants to be political, yet maintain the charisma of the underground (that is, by being non-political).It wants to be a commodity.… … Eine Kultur ohne Attribut ist Betrug. Die Kunst versucht, Wissenschaft zu sein.Sie beschäftigt sich mit Philosophie, Soziologie, Psychologie und möchte eine Quelle für die Pädagogik sein.Die Kunst will politisch sein, aber sich das Charisma des unpolitischen Undergrounds erhalten.Sie will Ware sein.