TRTRL: A Localized Resource-Efficient Learning Algorithm for Recurrent Neural Netowrks
Danny Budik, I. Elhanany · Conference proceedings · 2006
This paper introduces an efficient, low-complexity online learning algorithm for recurrent neural netowrks. The approach is based on the real-time recurrent learning (RTRL) algorithm, whereby the sensitivity set of each neuron is reduced to weights associated either with its input or output links. As a consequence, storage requirements are reduced from O(N3) to O(N2) and the computational complexity is reduced to O(N2). Despite the radical reduction in resource requirements, it is shown through simulation results that the overall performance degradation in rather minor. Moreover, the scheme lends itself to parallel hardware realization by virtue of the localized property that is inherent to the approach.