Universal Estimation of Information Measures for Analog Sources
Qing Wang, Sanjeev R. Kulkarni, Sergio Verdú · Foundations and Trends® in Communications and Information Theory · 2009
This monograph presents an overview of universal estimation of information measures for continuous-alphabet sources. Special attention is given to the estimation of mutual information and divergence based on independent and identically distributed (i.i.d.) data. Plug-in methods, partitioning-based algorithms, nearest-neighbor algorithms as well as other approaches are reviewed, with particular focus on consistency, speed of convergence and experimental performance.