A Sparsifier Model for Efficient Information Retrieval
Vyacheslav Dobrynin, Mark Sherman, Roman Abramovich, Alexey Platonov · 2024
The constant development of dense neural models leads to improved search quality. It is crucial to adapt these models to meet performance requirements. Solutions like SPLADE or SparseEmbed address this by solving the ranking task, whereas our work proposes addressing the simplified task of sparsifying dense vector representations. This approach facilitates the faster adaptation of new dense models for use with efficient inverted indexes. The importance of the independence property for sparse space features, achieved through the use of iVAE, is demonstrated. Additionally, the model is trained to maintain the ranking properties of the dense model, which in our case was a BERT model. As a result, the obtained model showed search quality close to the original BERT model. The proposed sparsification approach can be applied to other tasks requiring sparse spaces by adding new or replacing existing properties of the sparse space. Thus, the paper describes the main aspects of a sparsifier model applied to the task of information retrieval.