Bridge the Gap: Industrial Data Pipeline oriented Recommendation Enhancement
Jie Dai · 2021
In a world of overloading choices, recommender system has become indispensable as personalized information filter, which makes it of great interest to both industry and academia. Recently, inspiring insights are provided by deep learning based algorithms, and bring us many state-of-the-art recommender systems [1]. However, there exists a widening gap between researchers and engineers: academics usually prefer end-to-end recommender systems, whereas engineers tend to design a recommender system as a three stage pipeline, including preprocessing original data, recalling a candidate subset, and ranking recommendation results. Such pipeline can greatly benefit system development and project management, which has been proven by many Internet companies over recent years. To bridge the gap between end-to-end and pipeline, we propose three research directions and relevant methodologies, including label propagation based methods for preprocessing, graph neural network based negative sampling strategy for recall, and graph-based model of implicit interactions for ranking. In the following, we will describe them in detail.