Accessing the null space with nonlinear multilayer neural networks
Shelly D. D. Goggin, Karl E. Gustafson, Kristina M. Johnson · Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE · 1992
Nonlinear multilayer neural networks have been successful in solving problems in object recognition and decision making, which cannot be solved with nonlinear decision functions. These problems require the construction of a one-to-one or many-to-one mapping of input vectors to output vectors. If a finite training set is used, this mapping is a transformation between the set of values for the input elements to the set of values for the output elements. If the set of values for the input elements lies in the same subspace as the set of values for the output vectors, then a linear transformation can be made. Otherwise, either a neural network or some other nonlinear function is needed to construct the transformation. Nonlinear decision functions can make the transformation to sets of values for the output elements that are in a different subspace, but not an arbitrary subspace. The nonlinear multilayer neural network can make any transformation between sets of values of input elements and output elements, if enough hidden units are available. An explanation for the neural network's power to access any space, including the null space, is presented, along with some examples of the applicability of this result.