Information processing in negative feedback neural networks
Mark D. Plumbley · Network Computation in Neural Systems · 1996
Information theory suggests that extraction of the principal sub-space from data is useful when the input to a neural network is corrupted with additive noise. A number of neural network algorithms exist which can find this principal sub-space, many of which also extract the principal components of the input. However, when there is noise on both input and output of a network, simply extracting the principal sub-space (or components) is not sufficient to optimize information capacity. An approximate solution to maximizing information capacity would be to extract the principal sub-space of components with variances above a certain threshold, and then ensure that these are uncorrelated and that they have equal variance at the output. A neural network is described which uses negative feedback connections to achieve this uncorrelated, equal-variance solution.