Nonproduct Data-Dependent Partitions for Mutual Information Estimation: Strong Consistency and Applications
Jorge F. Silva, Shrikanth S. Narayanan · IEEE Transactions on Signal Processing · 2010
A new framework for histogram-based mutual information estimation of probability distributions equipped with density functions in (Rd,B(Rd)) is presented in this work. A general histogram-based estimate is proposed, considering nonproduct data-dependent partitions, and sufficient conditions are stipulated to guarantee a strongly consistent estimate for mutual information. Two emblematic families of density-free strongly consistent estimates are derived from this result, one based on statistically equivalent blocks (the Gessaman's partition) and the other, on a tree-structured vector quantization scheme.