Practice on Effectively Extracting NLP Features for Click-Through Rate Prediction
Hao Yang, Ziliang Wang, Weijie Bian, Yifan Zeng · 2023
Click-through rate (CTR) prediction is critical for industrial applications such as recommendation system and online advertising. Practically, there are a series of research proved that Natural Language Processing (NLP) features are helpful to improve CTR task performance. As these works show, there are different ways to extract NLP features. For example, keywords of item title are extracted as open-box feature by term frequency?inverse document frequency (tf-idf) method while item semantic embedding is extracted as black-box feature by shallow models (\emphe.g., word2vec) or deep learning models (e.g., BERT). However, these NLP models are pre-trained on NLP task, which is very different from the CTR task. Then it leads to the limited improvement of Area Under the ROC Curve (AUC) in CTR task. On the other hand, traditional NLP models for CTR task only consider open-box feature or black-box feature separately, which also leads to the discounted effect. Lastly, many NLP models are mainly used to extract semantic features only on item side. These methods take little account of user side information, or only IDs related features (\emphe.g., item's IDs) in user behavior sequence are introduced.