TRAINS-96 System Evaluation.
Amanda J. Stent, James F. Allen · 1997
In this report we describe an experiment designed to: ffl evaluate the performance of the TRAINS-96 system as a whole ffl examine the utility of a new robust post-parser module, recently added to the TRAINS system ffl explore the benefit to the user of receiving system feedback on speech input The evaluation uses the same task-based methodology as was used for the TRAINS-95 evaluation [7], in which the user and computer cooperatively solve a given problem. Success is measured in terms of task performance measures such as time to completion of a task, and the quality of the final plan produced. 1 1 This work was supported in part by the Office of Naval Research under research grant no. N00014-95-1-1088, and by the National Science Foundation under grant no. IRI-9623665. 1 Introduction TRAINS-96 is an extension of the TRAINS-95 system, part of "a long-term effort to develop an intelligent planning assistant that is conversationally proficient in natural language" [7]. The domain ...