Segmental HMM-based Part-of-speech tagger
Mohammad Hadi Bokaei, Hossein Sameti, Mohammad Bahrani, Bagher BabaAli · 2010
This paper presents a solution in order to solve the problem of using HMM-based POS tagger in some languages where a word can be comprised of several tokens. Viterbi algorithm is modified in order to support segment of words within a model state. In the other word, the proposed system has a built-in tokenizer where indicates words boundaries as well as its corresponding tag sequence.