Multi-scale dynamic neural net architectures
Les Atlas, Robert J. Marks, Michael O' Donnell, John Taylor · 2003
The design of specialized trainable neural network architectures for temporal problems is described. Multilayer extensions of previous dynamic neural net architectures are considered. Two of the key attributes of these architectures are smoothing and decimation between layers. An analysis of parameters (weights) to estimate suggests a massive reduction in training data needed for multiscale topologies for networks with large temporal input windows. The standard back-propagation training rules are modified to allow for smoothing between layers, and preliminary simulation results for these new rules are encouraging. For example, a binary problem with an input of size 32 converged in three iterations with smoothing and never converged when there was no smoothing.>