Fast Multidimensional Entropy Estimation by $k$-d Partitioning
Dan Stowell, Mark D. Plumbley · IEEE Signal Processing Letters · 2009
We describe a nonparametric estimator for the differential entropy of a multidimensional distribution, given a limited set of data points, by a recursive rectilinear partitioning. The estimator uses an adaptive partitioning method and runs in Theta(NlogN) time, with low memory requirements. In experiments using known distributions, the estimator is several orders of magnitude faster than other estimators, with only modest increase in bias and variance.