Hybrid Inflectional Stemmer and Rule-based Derivational Stemmer for Gujarati
Kartik Suba, Dipti Jiandani, Pushpak Bhattacharyya · 2011
In this paper we present two stemmers for Gujarati- a lightweight inflectional stemmer based on a hybrid approach and a heavyweight derivational stemmer based on a rule-based approach. Besides using a module for unsupervised learning of stems and suffixes for lightweight stemming, we have also included a module performing POS (Part Of Speech) based stemming and a module using a set of substitution rules, in order to improve the quality of these stems and suffixes. The inclusion of these modules boosted the accuracy of the inflectional stemmer by 9.6 % and 12.7 % respectively, helping us achieve an accuracy of 90.7%. The maximum index compression obtained for the inflectional stemmer is about 95%. On the other hand, the derivational stemmer is completely rule-based, for which, we attained an accuracy of 70.7 % with the help of suffix-stripping, substitution and orthographic rules. Both these systems were developed to be useful in applications such as Information Retrieval, corpus compression, dictionary search and as pre-processing modules in other NLP problems such as WSD. 1.