Rejoinder: "Data augmentation for support vector machines"
Nicholas G. Polson, Steven L. Scott · Bayesian Analysis · 2011
We thank all the discussants for their insights and comments on the article.Due to the subject matter specialization, Bayesian Analysis has a more homogeneous readership than journals that cater to a more general audience, so it is not surprising to find substantial agreement among the discussants and ourselves.Of course, readers may be disappointed by the lack of blood-sport normally associated with discussion articles.We apologize for this, and promise to write a more provocative article in the future. Mallick et al.Mallick et al. rightly point out that our focus on posterior inference for model parameters is only indirectly related to the classification performance that typically interests SVM users.The simulations provided by Mallick et al. are a welcome correction to our omission.The simulations show that the SVM criterion can in fact reduce the misclassification error compared to probit regression.Many Bayesians (including us) approach support vector machines with a wary suspicion that they are simply logistic regression's poor, non-probabilistic cousin.Simulations like this are useful data exercises that should force us to update that viewpoint.We have replicated the simulations in Table 1 with logistic regression in place of probit.The logistic regression and the SVM were both run, using spike-and-slab priors, on the spam data set from Section 5. We used the algorithm from Tüchler (2008) for the logit model.