Nonconvex Regularizations for Feature Selection in Ranking With Sparse SVM
Léa Laporte, Rémi Flamary, Stéphane Canu, Sébastien Dejean, Josiane Mothe · IEEE Transactions on Neural Networks and Learning Systems · 2013
Feature selection in learning to rank has recently emerged as a crucial issue. Whereas several preprocessing approaches have been proposed, only a few have focused on integrating feature selection into the learning process. In this paper, we propose a general framework for feature selection in learning to rank using support vector machines with a sparse regularization term. We investigate both classical convex regularizations, such as ℓ1or weighted ℓ1, and nonconvex regularization terms, such as log penalty, minimax concave penalty, or ℓppseudo-norm with p1scheme to address nonconvex regularizations. We conduct intensive experiments on nine datasets from Letor 3.0 and Letor 4.0 corpora. Numerical results show that the use of nonconvex regularizations we propose leads to more sparsity in the resulting models while preserving the prediction performance. The number of features is decreased by up to a factor of 6 compared to the ℓ1regularization. In addition, the software is publicly available on the web.