Effects of Forward Context on Learning with Recurrent Networks
Laurens R. Leerink, Marwan A. Jabri · 1994
For sequence based learning it is reasonable to expect that increasing the amount of forward context provided to a network will improve classification performance. However, for the problem of word boundary detection from continuous speech, we show that for every recurrent network architecture there exists an optimal amount of forward context. Results indicate that with increasing forward context conflicts appear between the storage of relevant contextual information and storage of intermediate results. Based on these arguments, recurrent units with variable discrete delays are proposed. Simulation results show that this architecture both increases performance and improves learning speed.