On training with slope adaptation for feedforward NNs
Danilo P. Mandic, Igor R. Krcmar · 2002
Relationships between the learning rate /spl eta/ and the slopes /spl beta/ in the tanh activation function for a feedforward neural network (NN) are provided. The analysis establishes the equivalence in the static and dynamic sense between a referent and an arbitrary feedforward NN which helps to reduce the number of degrees of freedom in learning algorithms for NNs.