Multi-Digit Recognition with Convolutional Neural Network and Long Short-Term Memory
Xianghui Liu, Yancong Deng, Yingkai Sun, Yuting Zhou · 2018
The paper recognizes digit sequence using convolutional neural network (CNN)encoder and long short-term memory (LSTM)decoder architecture. When designing CNN encoder, dropout and batch normalization are applied to prevent model from overfitting the train set. For decoder, we applied LSTM units to deal with vanishing and exploding gradients. The training was implemented by maximizing the log likelihood. During the test time, the model has a sequence accuracy of 92.53% on Street View House Number (SVHN)Dataset. In contrast, we also train other two models using Histogram of Oriented Gradient (HOG)as encoder and parallel dense layers as decoder respectively. By visualizing encoded features and comparing decoder's performance, we demonstrate the advantage of CNN regarding to image encoding and LSTM's advantage on sequence prediction.