Improving the discriminative power of inferred content information using segmented virtual profile
Haishan Liu, Anuj Kumar Goyal, Trevor Walker, Anmol Bhasin · 2014
We present a novel component of a hybrid recommender system at LinkedIn, where item features are augmented by a virtual profile based on observed user-item interactions. A virtual profile is generated by representing an item in the user feature space and leveraging the overrepresented user features from users who interacted with the item. It is a way to think about Collaborative Filtering with content features. The core principle is that if the feature occurs with high probability for the users who interacted with an item (henceforth termed as relevant users) versus those who did not (henceforth termed as non-relevant users), then that feature is a good candidate to be included in the virtual profile of the item in question. However, this scheme suffers from the data imbalance problem because observed relevant users are usually an extremely small minority group compared to the whole user base. Feature selection in this skewed setting is prone to noise from the overwhelming non-relevant examples that belong to the majority group. To alleviate the problem, we propose a method to select the most relevant non-relevant examples from the majority group by segmenting users on certain intelligently selected feature dimensions. The resulting virtual profile from the method is called the segmented virtual profile. Empirical evaluation on a real-world large scale recommender system at LinkedIn shows that our strategies for segmentation yield significantly better results.