Recommendations for All: Solving Thousands of Recommendation Problems Daily
Bhargav Kanagal, Sandeep Tata · 2018
Recommender systems are a key technology for many online services including e-commerce, movies, music, and news. Online retailers use product recommender systems to help users discover items that they may like. However, building a large-scale product recommender system is a challenging task. The problems of sparsity and cold-start are much more pronounced in this domain. Large online retailers have used good recommendations to drive user engagement and improve revenue, but the complexity involved is a roadblock to widespread adoption by smaller retailers. In this paper, we tackle the problem of generating product recommendations for tens of thousands of online retailers. Sigmund is an industrial-scale system for providing recommendations as a service. Sigmund was deployed to production in early 2014 and has been serving retailers every day. We describe the design choices that we made in order to train accurate matrix factorization models at minimal cost. We also share the lessons we learned from this experience - both from a machine learning perspective and a systems perspective. We hope that these lessons are useful for building future machine-learning services.