Method for Generalized Structured Information Extraction from Cards Based on Custom Template Matching
Kai Jiang, Tong Yang, Xue Li, Zizhong Wei, Qibin Chen, Qiang Duan, Junzheng Ge, Lei Zhang, Guangxin Wang, Jiahao Shao, Rui Li · 2023
Despite significant advancements in document recognition software catering to the needs of smart office environments, software in the card recognition domain remains scarce, often limited to single-function capabilities, with each software variant designed to recognize only one type of card. Card recognition confronts substantial challenges due to the diversity of card types, complexity of layouts, and the lower quality of images captured in natural settings. To address these challenges, this paper introduces a generalized method for structured information extraction from cards based on template matching. This method supports recognition of various common card formats and facilitates the extraction of structured data. We have incorporated an image enhancement technique based on conditional generative adversarial networks (GANs) to mitigate interference from irrelevant information and backgrounds, and to rectify text area distortions. A card positioning approach, predicated on anchor point matching, employs a flexible mechanism to align feature points between target and template images. Character recognition of card contents is executed using a convolutional neural network adapted for local adjustments, augmented by optimization steps including table detection, offset correction, and automated character rectification, thereby enhancing the quality of the outputted card structure information. Tests on cards collected from a multitude of natural scenes demonstrate that our generalized card recognition method can handle multiple card types simultaneously and maintain high matching and recognition rates. It exhibits strong practicality, especially in handling cards with complex features.