Evaluating an Information Extraction System
W. Lehnert, Claire Cardie, David A. Fisher, Joseph F. McCarthy, Ellen Riloff, Stephen Soderland · Integrated Computer-Aided Engineering · 1994
Many natural language researchers are now turning their attention to a relatively new task orientation known as information extraction. Information extraction systems are predicated on an I/O orientation that makes it possible to conduct formal evaluations and meaningful cross-system comparisons. T his article presents the challenge of information extraction and shows how information extraction systems are currently being evaluated. We describe a specific system developed at the University of Massachusetts, identify key research issues of general interest, and conclude with some observations about the role of performance evaluations as a stimulus for basic research.