Information geometry on ensemble HME model
Wen Jinwei, Luo Siwei, Huang Hua · 2004
An extendable framework is developed for an ensemble HME model based on the theoretical analysis of information geometry. In a hierarchical set of systems, a lower order system is included in the parameter space of a large one as a subset. Such a parameter space has rich geometrical structures that are responsible for the dynamic behavior of learning. The HME network divides a task into small tasks by the principle of divide and conquer to improve the performance of a single network. By studying the dual manifold architecture for mixtures of neural networks and analyzing the probability of knowledge-increasable model based on information geometry, the paper proposes a new method to achieve the multi-HME model that has knowledge-increasable and structure-extendible ability. The method helps to provide explanation of the transformation mechanism of the human recognition system and understand the theory of the global architecture of neural network.