Using a Large Set of EAGLES-compliant Morpho-syntactic Descriptors as a Tagset for Probabilistic Tagging
Dan Ioan Tufis · 2000
The paper presents one way of reconciling data sparseness with the requirement of high accuracy tagging in terms of fine-grained tagsets.For lexicon encoding, EAGLES elaborated a set of recommendations aimed at covering multilingual requirements and therefore resulted in a large number of features and possible values.Such an encoding, used for tagging purposes, would lead to very large tagsets.For instance, our EAGLES-compliant lexicon required a set of about 1000 morpho-syntactic description codes (MSDs) which after considering some systematic syncretic phenomena, was reduced to a set of 614 MSDs.Building reliable language models (LMs) for this tagset would require unrealistically large training data (hand annotated/validated).Our solution was to design a hidden reduced tagset and use it in building various LMs.The underlying tagger uses these LMs to tag a new text in as many variants as LMs are available.The tag differences between these variants are processed by a combiner which chooses the most likely tags.In the end, the tagged text is subject to a conversion process that maps the tags from the reduced tagset onto the more informative tags from the large tagset.We describe this processing chain and provide a detailed evaluation of the results.