A Novel Model for Chart-to-Text Generation by Utilizing NN Models
Hamad Munir Malik, Nabil M. Hewahi · 2024
This research introduces a novel neural network-based model designed to enhance the automatic generation of textual descriptions for scientific charts, building on the foundational work of “SCICAP: Generating Captions for Scientific Figures.” Our proposed model combines advanced Convolutional Neural Network (CNN) architectures for effective feature extraction from scientific charts with multiple Recurrent Neural Network (RNN) layers to generate descriptive, accurate captions. This study is distinguished due to its comparative analysis of different RNN architectures, such as Long Short-Term Memory (LSTM) and Gated Recurrent Units (GRU), to identify the most effective model for textual caption generation. Central to our methodology is using CNNs to parse visual input, transforming complex chart images into a rich set of feature vectors. These vectors serve as inputs for the sequential RNN layers, which construct coherent and contextually relevant textual descriptions. The performance of various CNN-RNN combinations is rigorously evaluated using the (Bilingual Evaluation Understudy (BLEU) metric, a standard in the natural language processing field for assessing the linguistic quality of machine-generated text against human-written references. Our research showed that Bi-LSTM performed best with a BLEU score of 0.3, suggesting that future research will require more reliable captioning.