Fusion of Human Decision and Artificial Intelligence in Optical Character Recognition through Mobile Phone Interaction
Yumeng He, Lingxiao Yang · 2022
Optical character recognition (OCR) is widely used for text recognition in various areas. It digitizes enormous texts efficiently with relatively high accuracy. However, the accuracy is severely undermined when texts are deformed, stained, or under dim light. The paper adds human decision to the recognition process of the deep learning algorithm and enhances the algorithm’s robustness and accuracy. We developed a mobile phone system that utilized the mainstream OCR technology to get the top N candidates sorted by confidence level and then properly displayed the candidates to the original text picture screened by the mobile phone camera. The final result is selected by a human finger tap on the right candidate. We conducted a first user study to compare the efficiency and accuracy between manual recognition and the developed system under several classical complex conditions, including stained, blurred, and incomplete text. The experiment results indicated that the system significantly improved text recognition efficiency and is dramatically faster than the manual recognition system. To explore more insights into our system, we conducted another user study to examine the influence of familiarity and encoding length. The results indicate that familiarity significantly influences recognition efficiency and correctness, but increasing length will actually increase the time spent on recognition.