Modeling Parameter Interactions in Ranking SVM
Yaogong Zhang, Jun Xu, Yanyan Lan, Jiafeng Guo, Maoqiang Xie, Yalou Huang, Xueqi Cheng · 2015
Ranking SVM, which formalizes the problem of learning a ranking model as that of learning a binary SVM on preference pairs of documents, is a state-of-the-art ranking model in information retrieval. The dual form solution of Ranking SVM model can be written as a linear combination of the preference pairs, i.e., w = ∑(i,j) αij (xi - xj), where αij denotes the Lagrange parameters associated with each pair (i,j). It is obvious that there exist significant interactions over the document pairs because two preference pairs could share a same document as their items. Thus it is natural to ask if there also exist interactions over the model parameters αij, which we may leverage to propose better ranking model. This paper aims to answer the question. Firstly, we found that there exists a low-rank structure over the Ranking SVM model parameters αij, which indicates that the interactions do exist. Then, based on the discovery, we made a modification on the original Ranking SVM model by explicitly applying a low-rank constraint to the parameters. Specifically, each parameter αij is decomposed as a product of two low-dimensional vectors, i.e., αij = vi, vj, where vectors vi and vj correspond to document i and j, respectively. The learning process, thus, becomes to optimize the modified dual form objective function with respect to the low-dimensional vectors. Experimental results on three LETOR datasets show that our method, referred to as Factorized Ranking SVM, can outperform state-of-the-art baselines including the conventional Ranking SVM.