Unified Embedding Based Personalized Retrieval in Etsy Search
Rishikesh Jha, Siddharth Subramaniyam, Ethan Benjamin, Thrivikrama Taula · 2024
Embedding-based neural retrieval is a prevalent approach to addressing the semantic gap problem, which often arises in product searches with tail queries. In contrast, popular queries typically lack context and have a broad intent where additional context from users' historical interaction can be helpful. In this paper, we share our novel approach to address both: semantic gap problem with an end-to-end trained model for personalized semantic retrieval. We propose learning a unified embedding model incorporating graph, transformer and term-based embeddings end-to-end and share our design choices for optimal tradeoff between performance and efficiency. We share our learnings in feature engineering, hard negative sampling strategy, and application of transformer model, including a novel pre-training strategy and other tricks for improving search relevance and deploying such a model at industry scale. Our personalized retrieval model significantly improves the overall search experience, as measured by a 5.58% increase in search purchase rate and a 2.63% increase in site-wide conversion rate, aggregated across multiple A/B tests - on live traffic.