Neural Models for Full Text Search
Nick Craswell · 2017
A fundamental concern in search engines is to determine which documents have the best content for satisfying the user, based on analysis of the user's query and the text of documents. For this query-content match, many learning to rank systems make use of IR features developed in the 1990s in the TREC framework. Such features are still important in a variety of search tasks, and particularly in the long tail where clicks, links and social media signals become sparse. I will present our current progress, in particular three different neural models, with the goal of surpassing the 1990s models in full text search. This will include evidence, using proprietary Bing datasets, that large-scale training data can be useful. I will also argue that for the field to make progress on query-content relevance modeling, it may be valuable to set up a shared blind evaluation similar to 1990s TREC, possibly with large-scale training data.