Asymptotically Optimal Regularization in Smooth Parametric Models
Percy Liang, Guillaume Bouchard, Francis Bach, Michael I. Jordan · 2009
Many regularization schemes have been employed in statistical learning, where each is motivated by some assumption about the problem domain. In this paper, we focus on regularizers in smooth parametric models and present an asymptotic analysis that allows us to see how the validity of these assumptions affects the risk of a particular regularized estimator. In addition, our analysis motivates an algorithm for optimizing regularization parameters, which in turn can be analyzed within our framework. We apply our analysis to several examples, including hybrid generative-discriminative learning and multi-task learning. 1