Learning temporally correlated representations using lstms for visual tracking
Qiaozhe Li, Xin Bo Zhao, Kaiqi Huang · 2016
In this paper, we propose to learn object representations with inference from temporal correlation in videos to achieve effective visual tracking. Unlike traditional methods which perform feature learning either at image level or based on intuitive temporal constraint, we employ the recurrent network with Long Short Term Memory (LSTM) units to directly learn temporally correlated representations of the objects in long sequences. The recurrent network is pre-trained offline with auxiliary data and then online optimized to adapt to the target-specific object. A structured SVM is employed to account for the temporally correlated object appearance as well as distinguish the object from background distraction. Experiment results not only show that the appearance and dynamic patterns of the objects can be characterized via temporally correlated feature learning, but also demonstrate that the proposed tracking algorithm performs favorably against the state-of-the-art methods.