Blitz: A Preprocessor for Detecting Context-Independent Linguistic Structures 1
Boris Katz, Deniz Yüret, Jimmy Lin, Sue Felshin, Rebecca Schulman, Adnan Ilik · 1998
The flow of natural language is often broken by constructions which are difficult to analyze with conventional linguistic parsers. To handle these constructions, which include numbers, dates, addresses, etc., and, to a lesser extent, proper nouns, NL systems typically implement specialized new rules. This leads to a level of complexity which renders maintenance or improvement difficult. Analyzing and tokenizing these constructions with an independent preprocessor can alleviate the burden on already taxed systems. Because these constructions have highly regular forms, strict structure, and can be largely understood in the absence of context, it is possible to shift the burden of processing away from the primary parser, and onto a simpler, faster, non-linguistic preprocessor.