Undersmoothed Kernel Entropy Estimators
Liam Paninski, Masanao Yajima · IEEE Transactions on Information Theory · 2008
We develop a ldquoplug-inrdquo kernel estimator for the differential entropy that is consistent even if the kernel width tends to zero as quickly as 1/N, whereNis the number of independent and identically distributed (i.i.d.) samples. Thus, accurate density estimates are not required for accurate kernel entropy estimates; in fact, it is a good idea when estimating entropy to sacrifice some accuracy in the quality of the corresponding density estimate.