Recurrent Neural Network with Human Simulator Based Virtual Reality
Yousif Ismail Mohammed Al Mashhadany · InTech eBooks · 2012
IntroductionDuring almost three decades, the study on theory and applications of artificial neural network has increased considerably, due partly to a number of significant breakthroughs in research on network types and operational characteristics, but also because of some distinct advances in the power of computer hardware which is readily available for net implementation.In the last few years, recurrent neural networks (RNNs), which are neural network with feedback (closed-loop) connects, have been an important focus of research and development.Examples include bidirectional associative memory (BAM), Hopfield, cellular neural network (CNN), Boltzmann machine, and recurrent back propagation nets, etc.. RNN techniques have been applied to a wide variety of problems due to their dynamics and parallel distributed property, such as identifying and controlling the real-time system, neural computing, image processing and so on.RNNs are widely acknowledged as an effective tool that can be employed by a wide range of applications that store and process temporal sequences.The ability of RNNs to capture complex, nonlinear system dynamics has served as a driving motivation for their study.RNNs have the potential to be effectively used in modeling, system identification, and adaptive control applications, to name a few, where other techniques may fall short.Most of the proposed RNN learning algorithms rely on the calculation of error gradients with respect to the network weights.What distinguishes recurrent neural networks from static, or feedforward networks, is the fact that the gradients are time dependent or dynamic.This implies that the current error gradient does not only depend on the current input, output, and targets, but rather on its possibly infinite past.How to effectively train RNNs remains a challenging and active research topic.The learning problem consists of adjusting the parameters (weights) of the network, such that the trajectories have certain specified properties.Perhaps the most common online learning algorithm proposed for RNNs is the real-rime recurrent learning (RTRL), which calculates gradients at time (k) in terms of those at time instant (k-1).Once the gradients are evaluated, weight updates can be calculated in a straightf__Gorward manner.The RTRL algorithm is very attractive in that it is applicable to real-time systems.However, the two main drawbacks of RTRL are the large computational complexity of O(N 4 ) and, even more critical, the storage requirements of O(N 3 ), where N denotes the number of neurons in the network.