Dual-Cycle Deep Reinforcement Learning for Stabilizing Face Tracking
Congcong Zhu, Zhenhua Yu, Suping Wu, Hao Liu · 2019
In this paper, we propose a dual-cycle deep reinforcement learning (DCDRL) method for stabilizing face tracking. Unlike most existing face tracking approaches which require per-frame annotations and dense facial landmarks are usually quite costly to annotate manually, our DCDRL aims to learn a robust face tracking policy by only using weakly-labeled annotations that were sparsely collected from raw video data. Motivated by the fact that facial landmarks in videos are usually coherent along with the forward and backward playing orders, we formulate the face tracking problem as a dual-cycle Markov decision process (MDP) by defining two agents for the forward-cycle and the backward-cycle accordingly. Specifically, both agents reason with the MDP policies by interacting in tuples of states, state transitions, actions and rewards during the MDP processes. Moreover, we carefully design a consistency-check reward function to track along until the target and back again it should arrive the start position in the reverse order. With the designed function, each policy generates a sequence of actions to refine the tracking routing by accumulating the maximal scalar rewards. This typically enforces the temporal consistency constraint on consecutive frames for reliable tracking outcomes. Experimental results demonstrate the robustness of our DCDRL versus many severe challenging cases especially in uncontrolled conditions.