Learning by Minimizing Resources in Neural Networks
Pál Ruján, Mario Marchand · Complex Systems · 1989
We reformulate the problem of supervised learning in neu ral nets to include the search for a network with minimal resources . The information processing in feedforward networks is described in geometrical terms as the partitioning of the space of possible input configurations by hyperplanes corresponding to hidden units. Regu lar partitionings introduced here are a special class of partitionings. Corresponding architectures can represent any Boolean function using a single layer of hidden units whose number depends on the specific symmetries of the function. Accordin gly, a new class of pla ne-cutting algorithms is proposed that const ruct in polynomial time a custom made architecture implementing the desired set of inputj'ouput ex amples . We report the results of our experiments on the storage and rule-extraction abilities of three-layer perceptrons synthetized by a simple greedy algorithm. As expected, simple neuronal structures with good generalization properties emerge only for a strongly corre lated set of examples.