GVdoc - Graph-based Visual DOcument Classification
Fnu Mohbat, Mohammed Javeed Zaki, Catherine Finegan‐Dollak, Ashish Kumar Verma · 2023
The robustness of a model for real-world deployment is decided by how well it performs on unseen data and distinguishes between in-domain and out-of-domain samples.Visual document classifiers have shown impressive performance on in-distribution test sets.However, they tend to have a hard time correctly classifying and differentiating out-ofdistribution examples.Image-based classifiers lack the text component, whereas multimodality transformer-based models face the token serialization problem in visual documents due to their diverse layouts.They also require a lot of computing power during inference, making them impractical for many real-world applications.We propose, GVdoc, a graph-based document classification model that addresses both of these challenges.Our approach generates a document graph based on its layout, and then trains a graph neural network to learn node and graph embeddings.Through experiments, we show that our model, even with fewer parameters, outperforms state-of-the-art models on out-of-distribution data while retaining comparable performance on the in-distribution test set.