Conflict resolution using weighted rules in HFST-TWOLC
Miikka Silfverberg, Krister Lindén · DSpace repository (University of Tartu) · 2009
In this article we demonstrate a novel way to resolve conflicts in two-level grammars by weighting the rules.The rules are transformed into probabilistic constraints, which are allowed to compete with each other.We demonstrate a method to automatically assign weights to the rules.It acts in a similar way as traditional conflict resolution, except that traditionally unresolvable left-arrow rule conflicts do not cause lexical forms to be filtered out.The two-level lexicon and probabilistic twolevel grammar are combined using the new transducer operation weighted intersecting composition.The result is a weighted lexical transducer.To the best of our knowledge, this is the first time probabilistic rules have been used to solve two-level rule conflicts.The possible applications of probabilistic lexical transducers range from debugging flawed two-level grammars to computer-assisted language learning.We test our method using a twolevel lexicon and grammar compiled with the open source tools HFST-LEXC and HFST-TWOLC.