An Ensemble Model for Error Modeling with Pseudoinverse Learning Algorithm
Sibo Feng, Xiaodan Deng, Ping Guo, Bo Zhao, Qian Yin, Hongfeng Wang · 2019
In Bayesian theory, the maximum posterior estimator uses prior information to estimate the noise in the machine learning model by adding the regularization term. The regularization terms L1and L2correspond to Laplacian prior and Guassian prior, respectively. In existing deep learning models, in order to use the gradient descent optimization algorithm and achieve good results, most models take L2regularization as the regularization term of the network model to fit the complex Guassian noise. However in practice, the Laplace noise and the Guassian noise are both considered as data noise. For multi-layer perceptrons, the difficulty caused by adding L1and L2into the optimization function of the network is solved by proposing an ensemble model for error modeling through adopting the divide and conquer strategy. First, several base learners are trained to fit different noise distributions of data, then the final results can be obtained by taking the results of each base leaner as new data to train a meta leaner, and get the final results. Among them, coordinate regression method is used to solve L1loss, while the pseudo-inverse learning algorithm is employed to solve L2loss. Both methods are nongradient optimization algorithms. The comparison results of the model on several data sets show that the proposed ensemble model achieves better performance.