Time Variability While Training a Parallel Neural Net Network
Tina L. Seawell · Open Scholarship Institutional Repository (Washington University in St. Louis) · 1995
The algorithmic analysis, data collection, and statistical analysis required to isolate the cause of time variability observed while an Elman style recurrent neural network is trained in parallel on a twenty processor SPARCcenter 2000 is described in detail. Correlations of system metrics indicate the operating system scheduler or an interaction of kernel processes is the most probable explanation for the variability. Introduction Neural networks are a promising approach to nonlinear problems. One of the major drawbacks to practical neural networks is the time required to train the network. Parallelization of neural network training algorithms can decrease training time from months to weeks. Isolation of algorithm and system problems becomes more difficult when parallelism is used. Determining the source of a problem often becomes an exercise in statistical analysis. A serial neural network training system, Trainrec, was developed by Barry Kalman and Stan Kwasny [KK93]. The training s...