Dynamical consistent recurrent neural networks
H. G. Zimmermann, Ralph Grothmann, Anton Maximilian Schäfer, Christoph Tietz · Proceedings. 2005 IEEE International Joint Conference on Neural Networks, 2005. · 2006
Recurrent neural networks aretypically consid- eredasrelatively simple architectures, whichcomealong with complicated learning algorithms. Mostresearchers focus onthe improvement ofthesealgorithms. Ourapproach isdifferent: Rather thanfocusing onlearning andoptimization algorithms, weconcentrate onthedesign ofthenetwork architecture. Aswewill show, manydifficulties inthemodeling ofdynam- ical systems canbesolved witha pre-design ofthenetwork architecture. We willfocus onlarge networks withthetask ofmodeling complete highdimensional systems (e.g. financial markets) instead ofsmallsetsoftimeseries. Standard neural networks tendtooverfit like anyother statistical learning system. Wewill introduce anewrecurrent neural network architecture in whichoverfitting andtheassociated loss ofgeneralization abilities isnota majorproblem. We willenhance thesenetworks by dynamical consistency.