Entropy-Robust Estimation Methods for Probabilities of Belonging in Machine Learning Procedures
Yuri S. Popkov, Alexey Yu. Popkov, Yuri A. Dubnov, Alexander Yu. Mazurov · 2022
Chapter 6 considers a special case of Randomized Parametric Models: the models with nonrandomized parameters. Recall that the uncertain model parameters and measurement noises lie within given intervals. Therefore, they can be transformed into nonnegative auxiliary variables belonging to the nonnegative unit cube. These variables are interpreted as the probabilities that the original parameters and noises belong to the corresponding original intervals. In other words, the auxiliary variables (the probabilities of belonging) are nonrandomized, and hence a parameterized model outputs a vector or trajectory defined by the values of these probabilities. The machine learning algorithm to estimate the probabilities of belonging is stated as a constrained maximization problem of the information entropy on a set defined by the empirical balance equations. The properties of the machine learning algorithm depend on the class of parameterized models under consideration. If the model is linear, the machine learning algorithm is reduced to an entropy-linear programming problem. In the general case, it is reduced to an entropy-nonlinear programming problem.