Recognizing person interactions
Ivan Laptev, Cordelia Schmid · 2013
by Ivan Laptev and Cordelia Schmid, both INRIA research directors. Both teams are specialized in computer vision, in particular visual recognition. Topic: With the amount of videos growing rapidly, there exists a clear need for automatic video indexing. Many existing methods rely on low-level video content descriptors [4], which allow to index video categories, but do not analyze human actions in details. Detailed analysis of actions is necessary if we want to understand how people interact with the world, for example what objects define an action [3]. Here, we will focus on interactions between people, relevant for example when analyzing the behavioral development of young children [1] or human interactions in TV shows [2]. Objective: The goal of the project is to build on an existing human pose estimation method [5] and to track the pose in videos. By tracking people and their body parts throughout sequences, we obtain information about their interactions. Initial simple measures could be for example the relative motion of people, the orientation of faces and the proximity of body parts. More advanced techniques include the design of models for joint spatio-temporal variations of body parts. The challenges will include reliable detection of person tracks and design of discriminative features given imprecise estimates of human poses. The approach will be evaluated on the Multimodal Dyadic Behavior (MMDB) dataset [1] and the Oxford TV dataset [2]. Skills and profile: The student must have solid programming skills (the project involves pro-gramming in C) as well as solid mathematics knowledge (especially linear algebra and statistics).