Teaching Large Language Models for Automatic Generation of Grammatically Sound Sentences from Handwritten Scribbles
Ginehe Carrasco, Ignatius Kovalenko, Leonidas Blakeney, Nikolai Farthington, Alessandro Grimaldi · 2024
The ability to accurately convert unstructured, noisy input into grammatically coherent text has long posed a challenge across various automated systems. Introducing a novel approach that leverages Mistral, an open-source language model, this research significantly improves the model’s capacity to extract grammatically sound sentences from scribbled handwriting without requiring human intervention. The enhanced model integrates noise-resistant layers and custom decoding mechanisms, leading to measurable improvements in sentence reconstruction, grammatical accuracy, and robustness to input distortion. Quantitative results demonstrate substantial gains in BLEU scores and error reduction across multiple noise levels, while qualitative evaluations confirm the model’s superior handling of ambiguous input. These findings have wide-reaching implications for automated text extraction tasks, particularly in environments where unstructured input remains prevalent, such as handwritten document digitization and real-time transcription systems.