An EM-based algorithm for recurrent neural networks
S. Ma, Chuanyi Ji · 2002
A stochastic model is established for fully-connected recurrent neural networks with sigmoid units based on Gibbs distributions. The EM (expectation-maximization) algorithm with a mean field approximation is then applied to train recurrent networks through hidden state estimation. The resulting EM-based algorithm, which reduces training the original recurrent network to training a set of individual feedforward neurons, simplifies the original training process and reduces the training time.