An Unsupervised Model of Orthographic Variation for Historical Document Transcription
Dan Garrette, Hannah Alpert-Abrams · 2016
Historical documents frequently exhibit extensive orthographic variation, including archaic spellings and obsolete shorthand.OCR tools typically seek to produce so-called diplomatic transcriptions that preserve these variants, but many end tasks require transcriptions with normalized orthography.In this paper, we present a novel joint transcription model that learns, unsupervised, a probabilistic mapping between modern orthography and that used in the document.Our system thus produces dual diplomatic and normalized transcriptions simultaneously, and achieves a 35% relative error reduction over a state-of-the-art OCR model on diplomatic transcription, and a 46% reduction on normalized transcription.