Multiobject Tracking in Videos Based on LSTM and Deep Reinforcement Learning
Mingxin Jiang, Chao Deng, Zhigeng Pan, Lanfang Wang, Xing Sun · Complexity · 2018
Multiple‐object tracking is a challenging issue in the computer vision community. In this paper, we propose a multiobject tracking algorithm in videos based on long short‐term memory (LSTM) and deep reinforcement learning. Firstly, the multiple objects are detected by the object detector YOLO V2. Secondly, the problem of single‐object tracking is considered as a Markov decision process (MDP) since this setting provides a formal strategy to model an agent that makes sequence decisions. The single‐object tracker is composed of a network that includes a CNN followed by an LSTM unit. Each tracker, regarded as an agent, is trained by utilizing deep reinforcement learning. Finally, we conduct a data association using LSTM for each frame between the results of the object detector and the results of single‐object trackers. From the experimental results, we can see that our tracker achieves better performance than the other state‐of‐the‐art methods. Multiple targets can be steadily tracked even when frequent occlusions, similar appearances, and scale changes happened.