Investigating bias in non-parametric mutual information estimation
Jie Ping Zhu, Jean-Jacques Bellanger, Huazhong Shu, Régine Le Bouquin Jeannès · 2015
In this paper, our aim is to investigate the control of bias accumulation when estimating mutual information from nearest neighbors non-parametric approach with continuously distributed random data. Using a multidimensional Taylor series expansion, a general relationship between the estimation bias and neighborhood size for plug-in entropy estimator is established without any assumption on the data for two different norms. When applied with the maximum norm, our theoretical analysis explains experimental simulation tests drawn in existing literature. In the experiments, two different strategies are tested and compared to estimate mutual information on independent and dependent simulated signals.