A Review of Deep Learning for Video Captioning
Moloud Abdar, K. Meenakshi, Swaraja Kuraparthi, Farhad Pourpanah, Daniel McDuff, Mohammad Ghavamzadeh, Shuicheng Yan, Abduallah Mohamed, Abbas Khosravi, Erik Cambria, Fatih Porikli · IEEE Transactions on Pattern Analysis and Machine Intelligence · 2024
Video captioning (VC) is a fast-moving, cross-disciplinary area of research that comprises contributions from domains such as computer vision, natural language processing, linguistics, and human-computer interaction. VC aims to understand a video and describe it through natural language descriptors. It plays a crucial role in various applications, from improving accessibility features such as low-vision navigation to advancing video question answering, video retrieval, and content generation. In this survey paper, we present a comprehensive review of deep learning-based VC methods. First, we provide an overview of VC, including the problem formulation, evaluation metrics, training losses, and attention-based architectures. Then, we categorize VC methods into several categories, including attention-based architectures graph networks, reinforcement learning, adversarial networks, and dense video captioning, and discuss each category in detail. In addition, we review existing data sets for VC methods and provide a discussion of research gaps and future research directions. We hope that this survey serves as a guide for researchers in relevant fields.