ResViT: An Integrated Approach Using ResNet50v2 and Vision Transformer for Enhanced Bangla Handwritten Character Recognition
Kazi Tanvir, Md Sadi Al Huda, Md. Sayem Kabir, Rezwan Obayed, Md. Fahim Attef, Md. Asraf Ali · 2024
Bangla, also known as Bengali, serves as the official language of Bangladesh and is the second most widely spoken language in India, flourishing in the voices of 300 million people as a language that bridges hearts and cultures. This linguistic richness is accompanied by a diverse character system and a vibrant literary heritage. However, the transition to digital storage and the conversion of handwritten documents pose notable challenges. ResNet-50 v2, a more advanced version of the ResNet-50 design, was released to improve deep network training by adding identity mappings and residual blocks. The Vision Transformer (ViT) extends the Transformer model from natural language processing to computer vision, employing patch-based image division and self-attention mechanisms for recognizing the global context. This study utilizes a model that combines features from the ViT and ResNet50V2 architectures, incorporating backbones for image processing, global average pooling, feature concatenation, and custom top layers with SoftMax activation, which achieved an impressive 97.21% accuracy on the extensive BanglaLekha- Isolated Dataset containing 166,000 images.