A fixed point update for kernel width adaptation in information theoretic criteria
António R. C. Paiva, José Carlos Príncipe · 2010
This paper presents a fixed point update for adaptation of the kernel width parameter in information theoretic criteria. These criteria are typically non-parametric and require a kernel width parameter to be appropriately set. The kernel width sets the smoothing bandwidth for estimation of the probability distribution of the error and, consequently, affects the performance surface. Hence, adaptation of the kernel width allows for the criterion, and its performance surface, to be adjusted to changes in the signal distribution. It is shown that the proposed fixed point update converges faster and is more stable when compared to a gradient update, and has no parameters. Moreover, it can be simplified to achieve the same computational complexity as the stochastic gradient update.