Incorporating Cognitive Abilities into Web Search Re-ranking
Tung Vuong, Pritom Kumar Das, Tuukka Ruotsalo · ACM Transactions on Information Systems · 2025
Web search ranking models learn from human interactions to improve retrieval performance, but they are presently limited by their use of behavioral factors, such as click-through data or dwell time, that do not account for differences in their users’ cognition. However, it is well understood that users’ behavior varies according to their abilities in processing information, making inferences, and interacting with computing systems. As a result, researchers may miss opportunities to design ranking models that are optimized for their users’ cognitive abilities. To address this, we report an approach for search result re-ranking that incorporates cognitive ability information in the ranking model. We report extensive empirical in-the-wild experiments with data from simulated tasks and real-world tasks of 20 participants to measure, predict, and use these data to train search result re-ranking models. Our results demonstrate that cognitive ability data significantly improve the effectiveness of re-ranking models in simulated-task and real-world conditions, and that cognitive abilities can be predicted from regular user interactions without requiring separate cognitive testing for each user. In particular, the models show improved performance in predicting the position of the documents the users select during search sessions. Our findings show that search engines have significant potential to improve their ranking performance by accounting for users’ cognitive ability.