A Classification Method based on Local Information and Nearest Neighbor Entropy Estimation
Ivan Lazić, Gorana Mijatović, Marta Iovino, Tatjana Lončar-Turukalo, Luca Faes · 2024
In the framework of information decomposition, non-parametric entropy estimators play a crucial role in dissecting the dynamics of complex systems. These estimators, often reliant on algorithms employing estimators of the probability density or related quantities such as the local information content, offer potential benefits for Bayesian-type classifiers. This paper introduces the Local Information Classifier (LIC), which specifically assigns data points to classes by minimizing the predicted pointwise entropy for each class. Through a comparative analysis with the Gaussian Bayes (GB) classifier, we illustrate the effectiveness and simplicity of the LIC in two simple simulations, while also highlighting similarities between the two models. Afterwards, we apply the proposed classifier to a stress classification task involving cardiovascular variability time series measured in a resting state and during orthostatic and mental stress. There, the LIC demonstrated greater stability across the chosen feature set in comparison to the GB classifier, relying on a single-feature classification approach.