Optimization of the Architecture of Feed-forward Neural Networks with Hidden Layers by Unit Elimination
Anthony Neville Burkitt · Complex Systems · 1991
A method for red ucing t he numb er of uni ts in t he hidden layers of a feed-for war d neur al network is pr esent ed . Starting wit h a net t hat is oversize, t he red un dan t unit s in t he hidden layer are elimi na t ed by introducing an ad dit ional cost fun ct ion on a set of a uxilia ry linear resp onse uni t s . T he ex t ra cost fu nct ion enables t he a uxiliary units t o fuse t ogether the redun dant uni t s on the origin al network , and t he aux iliary units serve only as an int erm edi at e const ruct t hat vanishes whe n t he met hod converges. Nume rica l tests on t he P arity and Symmetry problem s illu strat e t he usefu lness of t his method in pr actice.