Basic Methodologies Used in NLP Area
Yanzhe Wang · 2020 IEEE 3rd International Conference on Automation, Electronics and Electrical Engineering (AUTEEE) · 2020
For a better understanding of how the neural network is used in natural language processing (NLP), this article lists several basic language models and some latest improvements in order to explain the development of NLP more clearly. In these models, the continue bag of words model (CBOW) and skip-gram model are two simple models with one layer. The CBOW model is used to generate or guess a target word by providing context, while the skip-gram model is used for generated other words by providing one word. For more complex models, the convolution neural network (CNN) and recurrent neural network (RNN) are widely used in the NLP area, and CNN is used for classification tasks, while RNN is more used in the translation area of NLP. Based on the RNN model, a new structure encoder and decoder is designed to solve sequence to sequence problem, and several latest improvements are based on this structure.