A Study of Recent Advancements in Deep Learning for Natural Language Processing
Aditya Raj, Rajni Jindal, Aditya Kumar Singh, Ajay Kumar Pal · 2023
Natural Language Processing has gained interests and progress over the last decade. Its success can be attributed to the recent advances made in the field of Deep Learning. This review provides a detailed overview of recent advancements in deep learning techniques specially in the last decade (2013–2023) used for Natural Language Processing, including the architectural changes and capability enhancements. In this review we discussed the advancements in application areas like language modeling, machine translation and dependency parsing, while also comparing the models presented in each application areas. We compared the language models against their size, parameters, deep learning technique, features and drawbacks; dependency parsing models against their unlabeled and labeled accuracy scores, and machine translation models against their BLEU scores for WMT'14 English-French and English-German tasks. We also explored the potential challenges and limitations in these areas. Finally, we reviewed the current state-of-the-art multimodal model like GPT-4 and discussed how these models are evolving towards a general form of artificial intelligence.