Neural processing of complex continual input streams
Felix A. Gers, Jürgen Schmidhuber · 2000
Long short-term memory (LSTM) can learn algorithms for temporal pattern processing not learnable by alternative recurrent neural networks or other methods such as hidden Markov models and symbolic grammar learning. Here, we present tasks involving arithmetic operations on continual input streams that even LSTM cannot solve. However, an LSTM variant based on "forget gates," has superior arithmetic capabilities and does solve the tasks.