Performance measures for ranked output retrieval systems
佳弘 相良 · Joho Chishiki Gakkaishi · 2002
ranked output retrieval systems such as WWW search engines become pop -ular ， development of reasonable measures to evaluate performance of such systems are increasingly needed ． However ， there are no satisfactory measures for the pur - pose since they lack the following f()ur requirements ． 1 ) Relevance Judgment of all the records in a database should be avoided ． 2 ) User -oriented relevance judgment is needed ． 3 ) Relevance measures should have multi -values 、 4 ) Appropriateness of ranking outputs can be judged ， Neither traditional precision and recall nor normalized recall and sliding ratio that are used to evaluate ranked output re 七rieval s ． ystem 血ll丘11 the above require - ments ． Theref()re ， measures based on a modified sliding ratio ， rank correlation analysis and correlation analysis aLre proposed in this paper ・To check the sound - ness of these measures ， we experimented with 23 -search examples and confirmed