An Introduction to Information Theoretic Learning, Part I: Foundations
Daniel G. Silva, Denis G. Fantinato, Jânio Canuto, Leonardo Tomazeli Duarte, Aline O. Neves, Ricardo Suyama, Jugurta Montalvão, Romis Attux · Journal of Communication and Information Systems · 2016
With the increasing number of machine learning problems that are out of the linear and Gaussian paradigm, information theoretic learning (ITL) rises as a research field that proposes a modeling method with a wealthier statistical treatment of the adaptation criterion. In the first part of this tutorial, we introduce the main concepts of ITL and a key set of estimators that enable the implementation of algorithms, in the context of a wider view independent of the differentiability property.