Model-Based Artificial Intelligence Architecture for Digitizing Handwritten Medication Error Reports
Mohamed Ayachi Brini, Hanae Touati, Rafika Thabet, Franck Fontanili, Marie-Hélène Cleostrate, Marie-Noëlle Cufi, Marc Pruski, Elyes Lamine · 2024
Optical Character Recognition (OCR) is extremely useful in various sectors for exploring massive archived data. This technology enables the digitization of printed and handwritten texts that are frequently present in the medical field. For instance, medication error (ME) reports were previously and still in some healthcare facilities written manually, this has led to the accumulation of numerous handwritten data that are unfortunately challenging to exploit. Their digitization through OCR allows extracting important data from these documents and using them to populate the database to implement future analysis techniques to optimize the medication error management process. This paper presents a transformer-based handwritten recognition architecture that employs the Transformer-Based Optical Character Recognition (TrOCR) model combined with image segmentation techniques. Although the TrOCR model provided by Microsoft performs reasonably well in handwritten recognition, it is limited to English text because its pretrained version was trained exclusively on English samples. This limitation is problematic for us, as our task involves digitizing French medication dictation errors. Additionally, its limitation to processing single-line text images impairs its ability to recognize paragraphs. To address these limitations, we will fine-tune the model on French handwritten data and integrate a single-line level segmentation technique, thereby overcoming these constraints. Therefore, the preliminary results from implementing our proposed architecture are promising for the digitization of medication error reports.