Spiral Recurrent Neural Network for Online Learning
Huaien Gao, Rudolf Sollacher, Hans‐Peter Kriegel · 2007
Abstract. Autonomous, self * sensor networks require sensor nodes with a certain degree of “intelligence”. An elementary component of such an “intelligence ” is the ability to learn online predicting sensor values. We consider recurrent neural network (RNN) models trained with an extended Kalman filter algorithm based on real time recurrent learning (RTRL) with teacher forcing. We compared the performance of conventional neural network architectures with that of spiral recurrent neural networks (Spiral RNN)- a novel RNN architecture combining a trainable hidden recurrent layer with the “echo state ” property of echo state neural networks (ESN). We found that this novel RNN architecture shows more stable performance and faster convergence. 1