Image To Latex with DenseNet Encoder and Joint Attention
Jian Wang, Yunchuan Sun, Shenling Wang · Procedia Computer Science · 2019
Mathematical formula structural analysis usually converts mathematical formulas in images into Latex codes. It has been named as Image2Latex by OpenAi. At present, many researchers use the model in the field of image captioning for image2latex and have achieved good results. In this paper, we propose some improvements to the baseline model which is a sequence-to-sequence model used in image caption. We improve the encoder by employing densely connected convolutional network (DenseNet) because it can strengthen feature extraction and facilitate gradient propagation. We propose to use a more effective joint attention mechanism which include both spatial attention and channel-wise attention to solve this problem. We conducted experiments on the dataset im2latex-100k. Experimental results showed that our model improved the performance of formula analysis.