On 'Deep Evaluation' for Individual Computational Grammars and for Cross-Framework Comparison
Lars Hellan · 2007
A rather difficult point in grammar engineering evaluation is how to test and compare for analytic adequacy. A test design for 'deep' grammars is here proposed, where a parse is considered valid only if the assignment of syntactic and semantic structures that it displays obey certain conditions. The set of grammatical sentences in the test suite is construed as leaf types in a construction ontology, where the top types introduce the discriminants according to which constructions are categorized. These discriminants conform to notions shared across linguistic frameworks, and the validity conditions are defined within a well-known space of analytic parameters. One may envisage that with such a design, a meeting point can emerge for comparing frameworks with regard to agreed-upon aspects of linguistic content, and individual grammars with regard to their analytic aims and actual achievements relative to the aims. In Phillip Pullman's His dark materials, humans in one of the worlds have their soul partly realized as a little animal always accompanying them, sharing their thinking and emotions, but still behaving partly as independent agents; they are called daemons.