An entropy estimator improving the bias error
Yasunari Yokota · Electronics and Communications in Japan (Part II Electronics) · 2006
Abstract The entropy estimation is adopted as one of the useful methods for analysis in various fields including signal processing and system identification. However, the generally used entropy estimator is actually not an unbiased estimator. In this paper, the bias in the entropy estimator, namely, the bias error, was approximately formulated using the Taylor expansion of the entropy function. This approximation formula was reorganized and the coefficients contained were estimated to derive an approximation formula expressed in the form of a power series in terms of the number of samples. The derived approximation formula for the bias error was subtracted from the entropy estimator and this difference was proposed as a new entropy estimator. With the proposed entropy estimator, since the bias error is offset, its influence is suppressed and compared to the conventional entropy estimator and the bias error was reduced by a statistically significant difference. © 2006 Wiley Periodicals, Inc. Electron Comm Jpn Pt 2, 89(8): 60–69, 2006; Published online in Wiley InterScience ( www.interscience.wiley.com ). DOI 10.1002/ecjb.20287