Considering Subjects and Scenarios in Large-Scale User-Centered Evaluation of a Multilingual Multimodal Medical Search System.
Liadh Kelly, Lorraine Goeuriot, Gareth J. F. Jones, Allan Hanbury · Arrow@dit (Dublin Institute of Technology) · 2012
Abstract. Khresmoi aims to provide a multilingual and multimodal search and access system for biomedical information and documents, which targets three classes of user. Two of these are groups with general medical interests: the general public and general medical practitioners. The other is an example clinician group with a specific expertise: radiologists. Khresmoi targets diverse users, with: varying native language, language skills, medical knowledge, information needs or querying behaviour. The system seeks to provide these groups with innovative and effective services for searching through the large amount of available medical information. It gathers several innovative technologies linked to medical information (e.g. information extraction and retrieval, machine translation) to provide a comprehensive tool adapted to all users. In parallel to the development of the search system, a global evaluation strategy is being designed to enable an assessment of the efficiency and effectiveness of the developed technologies and of the quality of the support provided to the target users. These evaluation plans consist of both empirical and user-centred evaluations. We