Divergence Estimation for Multidimensional Densities Via $k$-Nearest-Neighbor Distances
Qing Wang, Sanjeev R. Kulkarni, Sergio Verdú · IEEE Transactions on Information Theory · 2009
A new universal estimator of divergence is presented for multidimensional continuous densities based on$k$-nearest-neighbor ($k$-NN) distances. Assuming independent and identically distributed (i.i.d.) samples, the new estimator is proved to be asymptotically unbiased and mean-square consistent. In experiments with high-dimensional data, the$k$-NN approach generally exhibits faster convergence than previous algorithms. It is also shown that the speed of convergence of the$k$-NN method can be further improved by an adaptive choice of$k$.