Research on Character Vector Generation Method Based on Words and Context
Yiying Wang, Hongyun Ning · 2019
In order to solve the problem of character information limitation and the neglect of polysemy, a character vector generation method based on words and context is proposed. First of all, introduce the concept of context vector. Considering the fact that the same word has different meanings in different contexts, some even have the opposite meanings, therefore, to correctly understand the meaning of Chinese words, we must fully consider the context in which it is located. The sentence in which the character or word is located is expressed in the form of a vector called context vector. Secondly, construct the polysemous storage of words. Considering the importance of context and the polysemy of words, this paper expands the expression of the original word vector, and adds a context linked list for each word vector. The multiple semantics of words are stored in multiple context vectors in the linked list, so as to solve the polysemy problem of words. Finally, a character vector generation method based on words and context is proposed. The target character vector is calculated by using the word vectors of all words containing the target character and its context similarity. Meanwhile, attention mechanism is introduced. When calculating, the current character context and the word context are compared, according to the character and word context similarity to distribute weights, so that the words with more similar context make greater contribution to the generation of the character vector. This method not only add the word information to the character, but also distinguishes different semantics according to the context of the character, and makes all words participate in the calculation of the character vector according to different weights. The experiment uses the character vector evaluation method based on external tasks, taking the news title similarity task as an example, comparing the performance of the traditional CBOW method and the new words and context based character generation method in the task. And the result shows that the improved model performs better than the traditional CBOW model in this title similarity task.