GE NLToolset
Lisa F. Rau, George R. Krupka, Paul S. Jacobs, Ira Sider, Lois C. Childs · 1992
This paper reports on the GE NLTooLSET customization effort for MUC-4, and analyzes th e results of the TST3 and TST4 runs . INTRODUCTIO NWe report on the GE results from the MUC-4 conference and provide an analysis of system performance .In general, MUC-4 was a very successful effort for GE .The NLTooLSET, a suite of natural language tex t processing tools designed for easy application in new domains, proved its mettle, as we were quickly able t o integrate the changes from the MUC-3 to the MUC-4 task .On the positive side, MUC-4 provided a thorough, fair test of system capabilities, and allowed us t o implement and test new strategies within the context of a task-driven system .Once again, the methodolog y of testing on a real task, along with the benefit of a common corpus, has produced advances in the field a s well as highlighting certain new aspects of text interpretation .One surprise was that we continued to make improvements in sentence-level parsing and interpretation, while at the end of MUC-3 we had suspected tha t improvements in parsing would not yield substantial improvements to our overall performance .On the negative side, major and significant improvements were not easy to make .Although improving the accuracy and coverage of the core language parsing mechanism accounted for some percentage of ou r improvements, the remainder of the gain in score is attributable to increases in the accuracy of the templat e post-filtering and to many small, incremental enhancements and modifications to the existing system .Thes e "diminishing returns" continue to stand in the way of vastly improved system performance .Although ther e are some major problems (such as world knowledge, event-based reasoning, and reference resolution) tha t can be said to account for much of the remaining error in MUC, it is not clear that MUC is really measurin g progress toward solving these major problems so much as progress on the many minor problems that ar e more easily solved .