Evaluation of user model effectiveness by simulation
Shlomo Berkovsky, Ariel Gorfinkel, Tsvi Kuflik, Larry Michael Manevitz · 2007
Abstract. Accurate initialization of a User Model (UM) is important for every system that provides personalized services. However, there are systems where this initialization is critical yet no user data is available from prior interactions. For example, one in which both the repeat usage by a user is rare, and the total interaction on a single use is relatively limited. Evaluating the effectiveness of a User Model and a particular instance of such a model is never an easy task, particularly through user studies. Moreover, evaluation typically focuses on the usability of an entire system rather than the performance of a specific UM instantiation. In this paper we propose evaluating the quality of UMs via simulation and comparison to a "gold standard". This standard is an approximation of the user's ideal model. We will demonstrate this through a case study of a museum's visitor guide system implemented in the Hecht Museum at the University of Haifa, Israel