Some new PAC-Bayesian bounds and their use in selection of regularization parameter for linear SVMs
Puja Sahu, N. Hemachandra · 2018
The PAC-Bayesian bounds offer performance guarantees on generalization error for many learning algorithms. We first obtain new bounds that are applicable to binary classifiers by using suitable choices of a distance function between the expected true risk and expected empirical risk, the squared distance with an updated threshold and another approximation to KL divergence. We use the fact the KL divergence between the averaged true and empirical risks is a convex function of the true risk to devise a simple convergent scheme to compute the interval for averaged true risk. One of our bounds leads to a finite step algorithm to compute an approximate interval for averaged true risk. A major contribution is the use of these results to compute PAC-Bayesian bound based intervals for averaged true risk when the regularization parameter in SVM design is chosen as per given prior and posterior distributions.