A Study on Recognizing Handwritten Mathematical Expressions
Ashwin Ragupathy, Gariman Gangwani, R. Shrinivedhaa, D. Uma · 2024
Handwritten mathematical expressions play a critical role in numerous domains, encompassing education, engineering, and science. The advent of deep neural networks, specifically those employing attention mechanisms and encoder-decoder architectures, has led to the widespread utilization of online recognition for handwritten mathematical expressions. In this research, Convolutional Neural Networks (CNNs) are extensively studied for their efficacy in transforming hand-drawn mathematical expressions into the corresponding LATEX representations. To further improve handwritten expression recognition, we investigate deep neural network models, such as VGG-16 and analyse its performance. We also studied the integration of transformer-based decoders in the recognition process, which offers concise model architectures and enhanced attention to image features like, Attention Aggregation based Bi-directional Mutual Learning Network (ABM) to leverage complementary information from both left-to-right (L2R) and right-to-left (R2L) directions. Moreover, we also studied and implemented hand-written expression recognition, incorporating syntax information into an encoder-decoder network. The proposed method utilizes grammar rules for converting LATEX markup sequences into parsing trees, significantly improving recognition accuracy. The approach outperforms previous methods on benchmark data sets. We studied the behaviour of Syntax Aware network for multiple different classes of mathematical expression