Transductive Learning for Reading Handwritten Tibetan Manuscripts
Sivan Keret, Lior Wolf, Nachum Dershowitz, Eric J. Werner, Orna Almogi, Dorji Wangchuk · 2019
We examine the use case of performing handwritten character recognition (HCR) on a newly compiled collection of Tibetan historical documents, which presents multiple challenges, including inherent challenges such as image quality and the lack of word separation, and dataset challenges such as a lack of supervised training data. To tackle these challenges, we introduce an end-to-end unsupervised full-document HCR approach composed of unsupervised line segmentation and a convolutional recurrent neural network, trained using solely synthetic data. Various augmentations are applied to these synthesized images, and we compare the effect of each augmentation on the HCR results. Since we work on a collection of historical manuscripts, we can fit the model to the available test data. During training, our network has access to both the labeled synthetic training data and the unlabeled images of the test set, and we adapt and evaluate four different semi-supervised learning and domain adaptation approaches for transductive learning in HCR. We test our approach on a set of 167 images from the "Kadam" collection, containing 829 lines. We show that correct data augmentation is crucial for the success of HCR trained solely on synthetic data and that using an effective transductive learning approach drastically improves results.