On decomposing MLPs
Simon Mark Lucas, Z. Zhao, Gavin C. Cawley, P.D. Noakes · 2002
The benefits of decomposing the multilayer preceptron (MLP) for pattern recognition tasks are investigated. For the case of N classes, instead of using 1 MLP with N outputs, N MLPs, each with a single output are used. In practice, this allows the use of fewer hidden units than would be used in the single MLP. It is found that decomposing the problem in this way allows convergence in fewer iterations. Not only does one save on the number of iterations as well as the time per iteration, but it becomes straightforward to distribute the training over as many workstations as there are pattern classes. The speedup is then linear in the number of pattern classes, assuming as many processors as classes. For the case of more classes than processors, the speedup is linear in the number of processors. It is shown that on a difficult hand-written optical character recognition (OCR) problem, the results obtained with the decomposed MLP are slightly superior than those for the conventional MLP, and are obtained in a fraction of the time.>