Distributed Occlusion Reasoning for Tracking with Nonparametric Belief Propagation
Erik B. Sudderth, Michael Mandel, William T. Freeman, Alan S. Willsky · 2004
We describe a three–dimensional geometric hand model suitable for vi-sual tracking applications. The kinematic constraints implied by the model’s joints have a probabilistic structure which is well described by a graphical model. Inference in this model is complicated by the hand’s many degrees of freedom, as well as multimodal likelihoods caused by ambiguous image measurements. We use nonparametric belief propaga-tion (NBP) to develop a tracking algorithm which exploits the graph’s structure to control complexity, while avoiding costly discretization. While kinematic constraints naturally have a local structure, self– occlusions created by the imaging process lead to complex interpenden-cies in color and edge–based likelihood functions. However, we show that local structure may be recovered by introducing binary hidden vari-ables describing the occlusion state of each pixel. We augment the NBP algorithm to infer these occlusion variables in a distributed fashion, and then analytically marginalize over them to produce hand position esti-mates which properly account for occlusion events. We provide simula-tions showing that NBP may be used to refine inaccurate model initializa-tions, as well as track hand motion through extended image sequences. 1