Natural Regularization in SVMs
Nuria Oliver, Alex J. Smola · 2015
Recently the so called Fisher kernel was proposed by [6] to construct discriminative kernel techniques by using generative models. We provide a regularization-theoretic analysis of this approach and extend the set of kernels to a class of natural kernels, all based on generative models with density p(xj`), like the original Fisher kernel. This allows us to incorporate distribution dependent smoothness criteria in a general way. As a result of this analyis we show that the Fisher kernel corresponds to a L 2 (p) norm regularization. Moreover it allows us to derive explicit representations of the eigensystem of the kernel, give an analysis of the spectrum of the integral operator, and give experimental evidence that this may be used for model selection purposes.