Recognizing Medicine Names from Bangladeshi Handwritten Prescription Images using TrOCR
Mohammad Faiyaz Uz Zaman, Bushra Rahman, Afsana Rubyat, Nabarun Halder, Asif Mahmud, Ashraful Islam, M. Ashraful Amin · 2024
Handwritten prescriptions are widely used in almost every healthcare facility which imposes a major drawback. These prescriptions are often difficult to understand, resulting in the wrong interpretation of medication information, leaving the health of patients at risk. This research study illustrates methods to extract medication information from handwritten prescriptions utilizing a straightforward, deep learning model called TrOCR: Transformer-Based Optical Character Recognition. This pre-trained model can detect complex handwritten language accurately by integrating the Optical Character Recognition (OCR) technique with Transformer architecture. In this study, we utilized a publicly available dataset containing 893 annotated prescription images to train and test the model. The OCR performance of our trained model on the test set showed a Character Error Rate (CER) of 16% and a Word Error Rate (WER) of 36%. With the integration of the Levenshtein distance-based text correction technique, the CER and WER dropped significantly to 7% and 11%, indicating improvement in the recognition of medication information.