A Data Analysis and Modelling Framework for the Evaluation of Interactive Information Retrieval
Ralf Bierig, Michael Cole, Jacek Gwizdka, Nicholas J. Belkin · Maynooth University ePrints and eTheses Archive (Maynooth University) · 2010
Fig. 1. System components of the data analysis and modeling framework Over the last two decades, Interactive IR (IIR) has established a new direction within the long tradition of IR that introduces the user at its center and poses new challenges for system evaluation. IR systems can improve performance by utilizing information about the entire interactive process of search. This approach has so far only been initially explored [1, 2] with much potential for the future. This demonstration presents an extensible data analysis and modeling framework that enables researchers to integrate, explore and analyze interactive experiment data obtained from task-based IIR experiments and build and test models of interactive user behavior. Figure 1 shows the framework components: The Event Representation integrates experiment data through the Event Reader Interface through a configurable set of Event Reader Import Rules into a unified and extensible event data structure. An extensible list of event types ensures that researchers can adapt and extend the framework to process data from a variety of IIR experiments on a single platform. Data Segmentation divides experiment data into semantic units guided by research hypotheses. A segmentation can for example differentiate