Learning Sparse Lexical Representations Over Specified Vocabularies for Retrieval
Jeffrey M. Dudek, Weize Kong, Cheng Li, Mingyang Zhang, Michael Bendersky · 2023
A recent line of work in first-stage Neural Information Retrieval has focused on learning sparse lexical representations instead of dense embeddings. One such work is SPLADE, which has been shown to lead to state-of-the-art results in both the in-domain and zero-shot settings, can leverage inverted indices for efficient retrieval, and offers enhanced interpretability. However, existing SPLADE models are fundamentally limited to learning a sparse representation based on the native BERT WordPiece vocabulary.