Towards new measures of information retrieval evaluation
William Hersh, Diane L. Elliot, David H. Hickam, Stephanie L. Wolf, Anna Molnár · 1995
All of the methods cumently used to evaluate information retrieval (Ill) systems have limitations in their ability to measure how well users are able to acquire information.We utilized on approach to assessing information obtained based on the user's ability to answer qtrcstions from a shortanswer test.Senior medical students took the ten-question test and then searched one of (WO IR systems on the five questions for which they were least ccrtuin of their answer.Our results showed thtit pre-searching scores on the test were low but that searching yielded a high proportion of answers with both systems.These methods are able to measure information obtained, and will be used in subsequent studies to assess differences among IR systems.provide a starting point at determining the quantity of useful information ob[ained from an IR system.they say Pemlission to nmke digil:il/h:lrd copies u 1'all or port i) 1'tl)is m:ttcri:ll without fee is grontcd provided [Il:it the co}>ics :irc not ll]:l(jc or distributed for profit or commercini ~LiV:lllhgC, lllc ACM copyriglll/ server notice, the tillc of lhc pu[)]ic?:ltion Jnd i(s d;i[u :tppc:tr, :111'j notice is given that copyright is by permission of tllc Assi)ci:ltiol] for Computin