Word Delimitation Issues in UD Japanese
Mai Omura, Aya Wakasa, Masayuki Asahara · Institutional Repositories DataBase (IRDB) · 2021
This article discusses word delimitation issues in Universal Dependencies (UD) Japanese.The Japanese language is morphologically rich and does not use white space to delimit words.Word delimitation is an important issue in the development of language resources.Even though UD defines the base unit word using syntactic words, UD Japanese utilises Short Unit Words (SUW), which are nearly the same as morphemes, the base unit word.We developed another word delimitation version of UD Japanese resources that uses Long Unit Words (LUW) as the base unit word, which can be regarded as syntactic words in Japanese.We then evaluated their reproducibility through publicly available language resources.The results show that the word delimitation and dependency structure of LUW-based UD Japanese reproduce the results using SUW-based UD Japanese.However, the lemmatisation of LUW is still more complex than that of SUW for a morphologically rich language.