Evaluating OCR-Based Retrieval Performance in Thai Academic Literature
Manot Mapato, Thanabordee Nammungkhun, Nattapong Karinta, Anurak Choosri, Worachet Pethaisong, Polapong Nawapongpipat · 2025
This research addresses the lack of comparative studies on Optical Character Recognition (OCR) methods for Thai academic documents, particularly in the context of Retrieval Augmented Generation (RAG) system integration. The study conducts a comparative analysis between Vision Language Models (VLMs) and traditional OCR systems, evaluating thirteen distinct models based on character recognition performance (CER, WER) and RAG integration efficiency (Retrieval Accuracy) across diverse pages of Thai academic documents, including abstracts, main content, mathematical equations, tables, and references. Findings reveal that VLMs significantly outperform traditional OCR systems; for instance, Gemini-2.5-flash-preview-04-17 achieved a CER of 15.5%, lower than the best traditional system, Azure OCR, at 33.0%. Notably, the study identified a non-linear relationship between character recognition errors (CER/WER) and retrieval performance, demonstrated by o4-mini achieving 98.9% Top 5k retrieval accuracy despite high CER (37.1%) and WER (49.4%). A Pareto Frontier analysis further indicated that Gemini-2.0-flash-lite, Gemini-2.5-flash-preview, and o4-mini offer the most favorable balance of cost and efficiency. This research provides valuable guidelines for selecting appropriate OCR technologies for Thai academic documents based on specific needs and suggests directions for future development to enhance access to Thai academic knowledge through advanced AI technologies.