Constraint-driven evaluation in UIMA Ruta
Andreas Wittek, Martin Toepfer, Georg Fette, Peter Klügl, Frank Puppe · 2013
Abstract. This paper presents an extension of the UIMA Ruta Workbench for estimating the quality of arbitrary information extraction models on unseen documents. The user can specify expectations on the domain in the form of constraints, which are applied in order to predict the F1 score or the ranking. The applicability of the tool is illustrated in a case study for the segmentation of references, which also examines the robustness for different models and documents. 1