Algorithm for Bayesian Networks Structure Learning Based on Information Entropy
Shaozhong Zhang, Xiukun Wang · Mini-micro Systems · 2005
The automated creation of Bayesian networks can be separated into two tasks, Structure learning, which consists of creating the structure of the Bayesian networks from the collected data, and parameter learning, which consists of calculating the numerical parameters for a given structure. A score function for optimization based on maximum mutual information entropy with odditional restriction is proposed. The entropy is based on KL distance, mutual information and maximum mutual information. A hill-climb algorithm is used in Bayesian networks structure learning. Some experimentation on K2, BB -MDL and MMI -L are given. The result indicates that the heuristic algorithm based MMI -L has more efficiency and precisely in cost and precision than K2 and BB -MDL.