Survey on Neural Networks in Natural Language Processing
Fei Hu · 2023
Neural Networks can learn natural language representations within levels of abstraction, and have recently shown much promise for Natural Language Processing (NLP) applications. Kinds of Neural Network Language Models (NNLMs) have gained a lot of success by predicting the next word given the other words in a context. These models have improved the state-of-the-art in word sense disambiguation, question answering, web search and many other domains in semantic parsing. Compared with "bag of word", Word Embedding has brought about breakthroughs in representations of words, sentences, or even documents. Recurrent Neural Network Language Models (RNNLMs) have shone light. They can learn and store long-memory knowledge that would help extracting better representations for long documents.