Extreme learning machine based on cross entropy
Yixin Cui, Junhai Zhai, Xizhao Wang · 2016
Extreme Learning Machine (ELM) is an algorithm for training single hidden layer feed-forward neural networks (SLFNs). Because ELM does not need the process of iterative learning, it is extremely faster than traditional learning algorithms such as back propagation algorithm and support vector machine. In ELM, the optimal solution with least squares norm is found by calculating the generalized inverse of hidden output matrix. When the order of hidden output matrix is a high, i.e., the number of hidden layer nodes is many, the over-fitting phenomenon will occur. Aiming at solving the over-fitting problem existing in ELM with many hidden layer nodes, this paper proposes a Cross Entropy based ELM (CE-ELM) in which, the mean square error minimization principle is replaced with the cross entropy minimization principle. The experimental results confirmed that the proposed CE-ELM can sufficiently overcome the drawback of overfitting in ELM with many hidden layer nodes.