Regression and Multiclass Classification Using Sparse Extreme Learning Machine via Smoothing Group L 1/2 Regularizer
Qinwei Fan, Lei Niu, Qian K. Kang · IEEE Access · 2020
Extreme learning machine (ELM) is a simple feedforward neural network, and it has been extensively used in applications for its extremely fast learning speed and good generalization performance. Nevertheless, it is implemented normally under the empirical risk minimization scheme and the model trained by ELM is prone to overfitting. In addition, the ELM provides more nodes than it actually needs, this means that the network structure is not sparse enough. To solve above problems, two efficient algorithms for training ELM: Group L1/2regularization and smoothing Group L1/2regularization methods are proposed in this article. But, the basic group L1/2regularization is nondifferentiable at the origin and which causes oscillation. So, we modify the basic group L1/2regularization by smoothing it at the origin. Simulation results show that the ELM with smoothing group L1/2regularization can effectively prune redundant nodes and redundant weights of the surviving nodes, which has better performance than traditional ELM, the ELM with L1regularization method, and with group L1/2regularization method.