Evaluation of Rich and Explicit Feedback for Exploratory Search
Esben Sørig, Nicolas Collignon, Rebecca Fiebrink, Noriko Kando · Goldsmiths Research Online · 2019
A user’s goals and interests during an exploratory search task are often ambiguous and complex. When engaging with new documents, people regularly use annotations to help better understand them and make them their own. These annotations can also pro- vide rich information to gain insight into a reader’s interests. In particular, it is possible to use highlights as a richer form of feed- back compared to traditional document-level relevance feedback. We first show that this form of feedback leads to improvements in document retrieval in exploratory search tasks with simulated users when compared to relevance feedback. We then present an evaluation platform which will allow us to understand the retrieval performance, user experience, and behavioral characteristics of human subjects using highlights as feedback. Finally, we propose an experimental design with human subjects. We hope that our experimental findings will help improve current simulated user evaluations for such systems.