The status of retrieval evaluation in the patent domain
Mihai Lupu · 2011
This paper is a result of the experience of the past three years in organizing evaluation campaigns for the patent domain. However, its aim is not really to give an account of what has been done in such evaluation campaigns, but rather to explore what has not been done. It also aims to convince all participants in this process, that we must have a more rigorous way of testing systems and system components. The efforts of the scientists, while being very narrow in the scope of each individual evaluation exercise, are still the most reliable way of comparing system components. Yet they fail to translate into an informative and comprehensible message for the users. I therefore propose a Patent Retrieval Tool Evaluation Map (PRET-EVAL map), to help all participants visualise the focus of each evaluation exercise.