A Statistical Approach to Language Modelling for the ATIS Problem
Joshua Koppelman · DSpace@MIT (Massachusetts Institute of Technology) · 1995
The Air Travel Information Service (ATIS) is the designated common task of the ARPA Spoken Language Systems Program. The specified task is to build and evaluate a system capable of handling continuous and spontaneous speech recognition as well as natural language understanding in the ATIS domain. The goal of this research is to develop an effective natural language component for the complete system, to answer queries posed through text input instead of speech. We limit our scope to deal only with those sentences which can be understood unambiguously out of context (the so-called "Class A" queries). Specifically, we wish to use the training data to assign a probability distribution to the reference interpretation, the NLParse, which will minimize the observed perplexity of our test data. The decoder component of the finished system will use the natural language probabilities to select the most probable NLParse translations for a given English input. The NLParse translation can then be unambiguously converted to SQL to find the correct answer.