Development of a test collection for studying long-term user modelling.
Frank Hopfgartner, Joemon M. Jose · eScholarship (California Digital Library) · 2011
Evaluating user modelling systems is a complex process and yet there currently exists no standard evaluation methodology.The interactive user evaluation paradigm cannot easily be applied since it does not allow users to follow their personal information need over a long time period.In this paper, we suggest a novel evaluation methodology for long-term user modelling which is based on user simulation.We first create individual ground truth data which we then use to simulate users interacting with a retrieval system.We also provide an example study exploiting these simulated profiles, illustrating how our method can be used to further study long-term user modelling techniques.