Generation and Comparison of Distributed Representations of Words from Japanese Wiktionary with BERT Sentence-Embedding
Ryota Nishiura, Seiji Tsuchiya, Hirokazu Watabe · International Journal of Service and Knowledge Management · 2024
In this paper, we introduce a novel approach to generate distributed representations of words.Our approach makes use of structured contents of Wiktionary, in contrast to existing methods such as Word2Vec, which are learned from unstructured corpora of plain text.Our approach generates distributed representations of headwords in Wiktionary by obtaining sentence embeddings of the corresponding contents of the entries with a pre-trained BERT model.We demonstrate that our proposed method outperforms a Word2Vec model in a XABC test, potentially due to its ability to leverage both the quality of a pre-trained BERT model and expert-moderated knowledge from Wiktionary.We also conducted a comparative study on a variety of different BERT models to find the model conditions most suitable for our purpose.