Error Entropy, Correntropy and M-Estimation
Weifeng Liu, Puskal P. Pokharel, J.C. Principe · Machine learning for signal processing ... · 2006
Minimization of the error entropy (MEE) cost function was introduced for nonlinear and non-Gaussian signal processing. In this paper, we show that this cost function has a close relation to a introduced correntropy criterion and M-estimation, thus it also theoretically explains the robustness of MEE to outliers. Based on this understanding, we propose a modification to the MEE cost function named minimization of error entropy with fiducial points, which sets the bias for MEE in an elegant and robust way. The performance of this new criterion is compared with the original MEE and the mean square error criterion (MSE) in robust regression and short-term prediction of a chaotic time series.