Using semantics to correct parser output for ATIS utterances
Sheryl R. Young · 1991
This paper describes the structure and operation of SOUL, or Semantically-Oriented Understanding of Language. SOUL is a knowledge intensive reasoning system which is opportunistically used to provide a more thorough, fine grained analysis of an input utterance following its processing by a case-frame speech parser. The SOUL postprocessor relies upon extensive semantic and pragmatic knowledge to correct, reject and/or clarify the outputs of the CMU PHOENIX case-frame parser for speech and speech transcripts. Specifically, we describe briefly both some of the linguistic phenomena which SOUL addresses and how SOUL works to correct inaccurate interpretations produced by the PHOENIX parser. Finally, we present the results on four separate, non-overlapping test sets. Our "pilot" test sets include the June 1990 DARPA ATISO test set and two test sets composed of unseen ATISO data distributed in June 1990 that, unlike the DARPA test sets, contain unrestricted utterances. Our forth test set is the official DARPA February 1991 ATIS1 test set. These evaluations illustrate the decrease in error rate that results from SOUL's semantic and pragmatic postprocessing are most pronounced in unrestricted, as opposed to carefully constrained test sets. Specifically, a performance comparison between unrestricted and restricted test sets in pilot experiments show that error rates are reduced by 84% as opposed to 54% when no utterances are pruned from the speaker transcripts.