A connection-limited neural network by InfoMax and InfoMin
Yoshitatsu Matsuda, Kazunori Yamaguchi · 2008
It is well known that edge filters in the visual system can be generated by the InfoMax principle. But, such models are nonlinear and employ fully-connected network structures. In this paper, a new artificial network model is proposed, which is based on the “InfoMin” principle and linear multilayer ICA (LMICA). This network utilizes cumulant-based objective functions which are derived from the InfoMax and InfoMin principles with large noise. Because the objective functions do not rely on any nonlinear models, a linear model can be employed. It simplifies the model considerably. Besides, this network can deal with quite large number of neurons by employing a connection-limited structure as in LMICA. In addition, it is more efficient than even LMICA because it does not need any prewhitening. Numerical experiments show that this network generates hierarchical edge filters from large-size natural scenes and verify the validity of the InfoMin principle.