Recurrent neural network weight estimation through backward tuning
Thierry Viéville, Xavier Hinaut, Thalita Firmo Drumond, Frédéric Alexandre · HAL (Le Centre pour la Communication Scientifique Directe) · 2017
We consider another formulation of weight estimation in recurrent networks,proposing a notation for a large amount of recurrent network units that helpsformulating the estimation problem. Reusing a “good old” control-theory principle,improved here using a backward-tuning numerical stabilization heuristic, we obtaina numerically stable and rather efficient second-order and distributed estimation,without any meta-parameter to adjust. The relation with existing technique is discussedat each step. The proposed method is validated using reverse engineeringtasks.