A New Method for the Construction of Evolving Embedded Representations of Words
Amal Bouraoui, Salma Jamoussi, Abdelmajid Ben Hamadou · 2017
There are several natural language processing tasks such as natural language understanding, information retrieval and social network analysis which require dynamic update of the word embeddings representation so as to keep pace with the words meanings evolving. Thanks to their ability to capture both syntactic and semantic information within language, the continuous bag-of-words, a learning word representation model, has gained a lot of attention from researchers and has been successfully applied to different NLP tasks. However, this model does not adequately capture the change of word semantics over time. This paper provides a new perspective to determine the semantic similarity of words over time by constructing several representations of words that reflect the semantic change of words based on latest embeddings. We examine the performance of our improved CBOW on an evolved data base collected from English Wikipedia pages. Our learned embeddings illustrate the semantic evolution of words better than the original CBOW.