VisualIE: Receipt-Based Information Extraction with a Novel Visual and Textual Approach
Hamza Gbada, Karim Kalti, Mohamed Ali Mahjoub · 2023
Information extraction (IE) from a receipt-based document requires understanding the contextual and visual semantics of texts. However, prior studies have predominantly utilized pre-trained language models and Optical Character Recognition (OCR) engines to extract textual features from the document and conduct entity extraction. Besides, this paper presents a novel approach for information extraction from receipt-based documents by leveraging both visual and textual features. The proposed approach employs a convolutional neural network to capture visual features and a word embedding model to capture textual features. These features are then combined and fed into a fully connected layer for entity classification. The experimental results demonstrate that the proposed approach outperforms the baseline and achieves state-of-the-art results in information extraction from receipts.