Scaling Factorization Machines with Parameter Server
Erheng Zhong, Yue Shi, Nathan J. Liu, Suju Rajan · 2016
Factorization Machines (FM) have been recognized as an effective learning paradigm for incorporating complex relations to improve item recommendation in recommender systems. However, one open issue of FM lies in its factorized representation (latent factors) for each feature in the observed feature space, a characteristic often resulting in a large parameter space. Therefore, training FM (in other words, learning a large number of parameters in FM) is a computationally expensive task. Our work targets to improve the scalability of FM by building it in a distributed environment. We propose a new system framework that integrates Parameter Server (PS) with the Map/Reduce (MR) framework. In addition to the data parallelism achieved via MR, our framework particularly benefits from PS for model parallelism, a critical characteristic for learning with a large number of parameters in FM. We further address two specific challenges in our system, namely, communication cost and parameter update collision. Through both offline and online experiments on recommendation tasks, we demonstrate that the proposed system framework succeeds in scaling up FM for very large datasets, while it also maintains competitive performance on recommendation quality compared to alternative baselines.