Large-scale Evaluation of Context Matching
Bernardo Magnini, Manuela Speranza · 2007
Context matching algorithms automatically discover semantic relations between two autonomously developed conceptual representations of two overlapping domains. Although significant work has been carried on the theoretical ground, the empirical evaluation of matching algorithms is still an open area for research. In this paper we introduce CTXMATCH, a matching algorithm which applies to symbolic based conceptual representations organized as a hierarchy of concepts. Typical examples of such conceptualizations are electronic market catalogs (e.g., UNSPSC and eCl@ss) and web directories (e.g., Google and Yahoo!). Then, we report on different evaluation experiments, that have been carried out with two main purposes: we wanted to evaluate the CTXMATCH algorithm in real, large scale scenarios and to test different evaluation methodologies. Results are to be considered as a first contribution toward shared evaluation practices and resources for context matching algorithms.