Application of Renyi Entropy and Mutual Informa-tion of Cauchy-Schwartz in Selecting Variables.
Leonardo Macrini, Leonardo Barroso Gonçalves · IWBBIO · 2013
This paper approaches the algorithm of selection of variables named MIFS-U and presents an alternative method for estimating entropy and mutual information, measures that constitute the base of this selection algorithm. This method has, for foundation, the Cauchy-Schwartz quadratic mutual infor- mation and the Renyi quadratic entropy, combined, in the case of continuous variables, with Parzen Window density estimation. Experiments were accom- plished with public domain data, being such method compared with the original MIFS-U algorithm, broadly used, that adopts the Shannon entropy definition and makes use, in the case of continuous variables, of the histogram density es- timator. The results show small variations between the two methods, what sug- gest a future investigation using a classifier, such as Neural Networks, to quali- tatively evaluate these results, in the light of the final objective which is greater accuracy of classification.