Using specific evaluation for comparing and combining competing algorithms: applying it to table column detection
Ana Costa e Silva · Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE · 2011
It is a commonly used evaluation strategy to run competing algorithms on a test dataset and state which performs better in average on the whole set. We call this generic evaluation. Although it is important, we believe this type of evaluation is incomplete. In this paper, we propose a methodology for algorithm comparison, which we call specific evaluation. This approach attempts to identify subsets of the data where one algorithm is better than the other. This allows not only knowing each algorithm's strengths and weaknesses better but also constitutes a simple way to develop a combination policy that allows enjoying the best of both. We shall be applying specific evaluation to an experiment that aims at grouping pre-obtained table cells into columns; we demonstrate how it identifies a subset of data for which the on-average least good but faster algorithm is equivalent or better, and how it then manages to create a policy for combining the two competing table column delimitation algorithms.